Split and Assemble large file (around 2GB) in C# dot net Programming

Hi friends after a long time I’m back again. Now with a quite different coding flavor in different area. I will express about some operation with File. I will give some code to split and assemble large file. These codes can split up to 2 GB (approximately) file to any number of small file (minimum 1 MB) and also can assemble these small files to get the original file. Be sure that in assembling all small files need to keep in a folder otherwise it will fail to make original file and throw an error.

To do this in C# dot net , I’ve taken help of System.IO namespace and BinaryReader and BinaryWriter class. To learn about these classes please see msdn site. Here I’ve follow very simple algorithm.

For slice a file steps are:
(i) Open a large file in read mode by binary reader stream.
(ii) Execute step iii and step v, until file read reach at end of file.
(iii) Read from that stream and set these bytes in byte array
(iv) Make a new file name from original file name with slice number, for last file slice add ‘E’ after slice number.
(v) Save these bytes from array to a new file with ‘File’ class’s ‘WriteAllBytes’ method.
(vi) close the dot net binary stream.

Download source code from here:
http://alap.me/blog/All_Source_Codes.rar

These code as follows –

(i) BinaryReader br=new BinaryReader(File.Open(filename, FileMode.Open));

(ii) while (br.BaseStream.Length > sliceLen * counter)

(iii) br.BaseStream.Read(buffer, 0, sliceLen);
(iv) curFileName = filename + "." + counter.ToString();
curFileName = filename + "." + counter.ToString() + ".E";
(v) File.WriteAllBytes(curFileName, buffer);
(vi) br.Close();

For assemble these files need to follow these steps –

(i) Create a binary writer stream and open a binary file in append mode.
(ii) Execute from step iii to v
(iii) Generate file name in runtime depends on pervious file name.
(iv) Check for last file slice, last file slice name ends with last character ‘E’. If present file is the last slice then exit from that loop.
(v) Read all bytes from file and set in a byte array then write these byte data by binary writer.
(vi) Close the binary writer.

These code as follows-

(i) BinaryWriter bw = new BinaryWriter(File.Open(orgFile, FileMode.Append))
(ii) while(true)
(iii) nextFileName = orgFile + "." + counter.ToString();
(iv) if (File.Exists(nextFileName + ".E"))

(v) buffer = File.ReadAllBytes(nextFileName + ".E");
bw.Write(buffer);
(vii) bw.Close();

I’ve not covered everything in algorithm but I believe that you will understand these easily.

The complete codes are given below:

// File cutter assembler in microsoft c# dot net
//This code has written by Suman Biswas in 2008.
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Text;
using System.Windows.Forms;
using System.IO;

namespace FileCutter
{
//This class is used to call the actual file operation class.
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
this.Text = "File cutter & assembler (upto 1.96 GB) by Suman Biswas";
}
FileHandling obj = new FileHandling();
private void btnSelectFile_Click(object sender, EventArgs e)
{

obj.SplitUp(SelectFile(),int.Parse(textBox1.Text));


}

private void button1_Click(object sender, EventArgs e)
{
obj.MargeUp(SelectFile());
}
private string SelectFile()
{
OpenFileDialog fbd = new OpenFileDialog();
if (fbd.ShowDialog() != DialogResult.OK)
{
MessageBox.Show("No file selected");
return "";
}
else
return fbd.FileName;
}
}

//Main file operation is done here.
class FileHandling
{
int sliceLen = 1024 * 1024;
int counter = 0;

public void SplitUp(string filename,int fileSizeInMB)
{
if (fileSizeInMB < slicelen =" 1024" counter =" 0;" buffer="new" br="new" slicelen =" (int)br.BaseStream.Length;"> sliceLen * counter)
{
if (br.BaseStream.Length > sliceLen * (counter + 1))
{
br.BaseStream.Read(buffer, 0, sliceLen);
curFileName = filename + "." + counter.ToString();
}
else
{
int remainLen = (int)br.BaseStream.Length - sliceLen * counter;
buffer = new byte[remainLen];
br.BaseStream.Read(buffer, 0, remainLen);
curFileName = filename + "." + counter.ToString() + ".E";
}

if (File.Exists(curFileName))
File.Delete(curFileName);

File.WriteAllBytes(curFileName, buffer);
counter++;
}
br.Close();
MessageBox.Show("File spilitted successfully");
}

public void MargeUp(string firstFileName)
{
if (firstFileName.Length < 1)
return;

string endPart = firstFileName;
string orgFile = "";

orgFile = endPart.Substring(0, endPart.LastIndexOf("."));
endPart = endPart.Substring(endPart.LastIndexOf(".") + 1);

if (endPart == "E")//If only one slice is there
{
orgFile = orgFile.Substring(0, orgFile.LastIndexOf("."));
endPart = "0";
}

if (File.Exists(orgFile))
{
if (MessageBox.Show(orgFile + " already exists, do you want to delete it", "", MessageBoxButtons.YesNo) == DialogResult.Yes)
File.Delete(orgFile);
else
{
MessageBox.Show("File not assembled. Operation cancelled by user.");
return;
}
}

//Assembling starts from here
BinaryWriter bw = new BinaryWriter(File.Open(orgFile, FileMode.Append));
string nextFileName = "";
byte []buffer=new byte [bw.BaseStream.Length];


int counter=int.Parse(endPart);
while(true)
{
nextFileName = orgFile + "." + counter.ToString();
if (File.Exists(nextFileName + ".E"))
{
//Last slice
buffer = File.ReadAllBytes(nextFileName + ".E");
bw.Write(buffer);
break;
}
else
{
buffer = File.ReadAllBytes(nextFileName);
bw.Write(buffer);
}
counter++;
}
bw.Close();
MessageBox.Show("File assebled successfully");
}

}

}

Synchronous Server Socket using Network Stream in C# .Net Programming

Hi frends,I’m back again with two sample socket application. These two project also for beginner’s in C#.Net Socket world. One of these is Server socket application and another is Client socket application. Both application has used NetworkStream class to send and receive data between Client and Server. These two has written based on previous articles on Socket programming and these are very easy to learn. I hope any one can understood these very easyly. If you feel any problem then please reply me via comment I will response these.

Here Server socket with Network stream in C# has written and in different blog post contains the client application. Both two has written based on synchronous communication mode, but in future I will show you, how you can built asynchronous socket application by these synchronous socket. So lets starts code.

using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Sockets;

namespace NetworkStreamSocketServer
{
class Program
{
static void Main(string[] args)
{
IPEndPoint ipEnd = new IPEndPoint(IPAddress.Any, 5656);
Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP);
sock.Bind(ipEnd);
sock.Listen(100);
Socket clientSock = sock.Accept();
byte[] clientData = new byte[1024];

//With other Socket server application difference only here.
//Create an Network stream object and wait for client's request
NetworkStream ns = new NetworkStream(clientSock);

//Wait for new connection and to receive data from client
int receivedBytesLen = ns.Read(clientData, 0, clientData.Length);
string clientDataInString = Encoding.ASCII.GetString(clientData, 0, receivedBytesLen);
ns.Flush();//Free Stream buffer
Console.WriteLine("Received Data {0}", clientDataInString);

string clientStr = "Client Data Received";
byte[] sendData = new byte[1024];
sendData = Encoding.ASCII.GetBytes(clientStr);
//Now network stream object send some data to client
ns.Write(sendData, 0, sendData.Length);
//Release all resources
ns.Close();
clientSock.Close();
Console.ReadLine();
}
}
}

Asynchronous Socket Client for Beginner

It's a sample client socket code that code based on MSDN sample code.

using System;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.Text;

// State object for receiving data from remote device.
public class StateObject
{
// Client socket.
public Socket workSocket = null;
// Size of receive buffer.
public const int BufferSize = 1024;
// Receive buffer.
public byte[] buffer = new byte[BufferSize];
// Received data string.
public StringBuilder sb = new StringBuilder();
}

public class AsynchronousClient
{
// The port number for the remote device.
private const int port = 5656;

// ManualResetEvent instances signal completion.
private static ManualResetEvent connectDone =new ManualResetEvent(false);
private static ManualResetEvent sendDone =new ManualResetEvent(false);
private static ManualResetEvent receiveDone =new ManualResetEvent(false);

// The response from the remote device.
private static String response = String.Empty;

private static void StartClient()
{
// Connect to a remote device.
try
{
// Establish the remote endpoint for the socket.
IPHostEntry ipHostInfo = Dns.Resolve("localhost");
IPAddress ipAddress = ipHostInfo.AddressList[0];
IPEndPoint ep = new IPEndPoint(ipAddress, port);

// Create a TCP/IP socket.
Socket client = new Socket(AddressFamily.InterNetwork,
SocketType.Stream, ProtocolType.Tcp);

// Connect to the remote endpoint.
client.BeginConnect(ep,
new AsyncCallback(ConnectCallback), client);
connectDone.WaitOne();

// Send test data to the remote device.
Send(client, "Data Send to local server.");
sendDone.WaitOne();

// Receive the response from the remote device.
Receive(client);
receiveDone.WaitOne();

// Write the response to the console.
Console.WriteLine("Response received : {0}", response);

// Release the socket.
client.Shutdown(SocketShutdown.Both);
client.Close();

}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}

private static void ConnectCallback(IAsyncResult ar)
{
try
{
// Retrieve the socket from the state object.
Socket client = (Socket)ar.AsyncState;

// Complete the connection.
client.EndConnect(ar);

Console.WriteLine("Socket connected to {0}",client.RemoteEndPoint.ToString());

// Signal that the connection has been made.
connectDone.Set();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}

private static void Receive(Socket client)
{
try
{
// Create the state object.
StateObject state = new StateObject();
state.workSocket = client;

// Begin receiving the data from the remote device.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,
new AsyncCallback(ReceiveCallback), state);
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}

private static void ReceiveCallback(IAsyncResult ar)
{
try
{
// Retrieve the state object and the client socket
// from the asynchronous state object.
StateObject state = (StateObject)ar.AsyncState;
Socket client = state.workSocket;

// Read data from the remote device.
int bytesRead = client.EndReceive(ar);

if (bytesRead > 0)
{
// There might be more data, so store the data received so far.
state.sb.Append(Encoding.ASCII.GetString(state.buffer, 0, bytesRead));

// Get the rest of the data.
client.BeginReceive(state.buffer, 0, StateObject.BufferSize, 0,new AsyncCallback(ReceiveCallback), state);
}
else
{
// All the data has arrived; put it in response.
if (state.sb.Length > 1)
{
response = state.sb.ToString();
}
// Signal that all bytes have been received.
receiveDone.Set();
}
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}

private static void Send(Socket client, String data)
{
// Convert the string data to byte data using ASCII encoding.
byte[] byteData = Encoding.ASCII.GetBytes(data);

// Begin sending the data to the remote device.
client.BeginSend(byteData, 0, byteData.Length, 0,new AsyncCallback(SendCallback), client);
}

private static void SendCallback(IAsyncResult ar)
{
try
{
// Retrieve the socket from the state object.
Socket client = (Socket)ar.AsyncState;

// Complete sending the data to the remote device.
int bytesSent = client.EndSend(ar);
Console.WriteLine("Sent {0} bytes to server.", bytesSent);

// Signal that all bytes have been sent.
sendDone.Set();
}
catch (Exception e)
{
Console.WriteLine(e.ToString());
}
}

public static int Main(String[] args)
{
StartClient();
Console.ReadLine();
return 0;
}
}

Server Socket Programming in C#, how to start

For beginner socket is quite complicated. But it's not very hard to learn. Basically I've learned it within few hours. So I believe it can learn any body. However let's start.

Socket programming have two part
I. Server
II. Client.

I. Server:
To start server need to follow some steps, these are:
i. First create a IPEndPoint
[e.g. IPEndPoint ipEnd = new IPEndPoint(IPAddress.Any, port);]

ii. Create a socket object.
[Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP);]

iii. Bind socket with IPEndPoint.
[sock.Bind(ipEnd);]

iv. Place socket in Listen mode (to accept call of client)
[sock.Listen(maxClientReceived);]

v. When any call comes from client Accept that call.
[Socket clientSock = sock.Accept();]
At that position Accept() return a new socket to continue communication with called client. By that socket Client-Server communication continue.

To send some data (in byte array form) to client from server just write
clientSock.send(byteArrayData);

To receive client data just write
int receivedLen= clientSock.Receive(clientData);
Receive() function reads data in byte array form from client socket and writes to 'clientData' array, and return integer value how much bytes has received.

I'm giving complete code of a simple server socket below:-


using System;
using System.Collections.Generic;
using System.Text;
using System.Net;
using System.Net.Sockets;
namespace beginSocketServer
{
class Program
{
static void Main(string[] args)
{
IPEndPoint ipEnd = new IPEndPoint(IPAddress.Any, 5656);
Socket sock = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.IP);
sock.Bind(ipEnd);
sock.Listen(100);
Socket clientSock = sock.Accept();
byte[] clientData = new byte[1024];
int receivedBytesLen = clientSock.Receive(clientData);
string clientDataInString = Encoding.ASCII.GetString(clientData, 0, receivedBytesLen);
Console.WriteLine("Received Data {0}", clientDataInString);
string clientStr = "Client Data Received";
byte[] sendData = new byte[1024];
sendData= Encoding.ASCII.GetBytes(clientStr);
clientSock.Send(sendData);
clientSock.Close();
Console.ReadLine();
}
}
}

To run that server code you don't need to write client code just open a webbrowser and write (I've use Mozilla Firefox 2.0.0.7) :
http://localhost:5656/

Server console output was:

Received Data GET / HTTP/1.1Host: localhost:5656User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7Accept: text/xml,application/xml,application/xhtml+xml,text/html;q=0.9,text/plain;q=0.8,image/png,
*/*;q=0.5Accept-Language: en-us,en;q=0.5Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7Keep-Alive: 300Connection: keep-alive

And in web browser output was:Client Data Received.

Ok, Simple server socket application has done. Next will give simple client socket application for beginner.


see more