Wednesday, February 20, 2008

How To Go Slow: Summing Arrays



In this entry I'll talk about thrashing when iterating over complex arrays.

Consider a square array with the size of N = 10000;
Now, consider the code that summs elements of the square array

for (int row = 0; row < N;, ++row)
for (int col = 0; col < N; ++col)
sum += A[row, col];

Or on the other hand:
for (int col = 0; col < N; ++col)
for (int row = 0; row < N; ++row)
sum += A[row, col];


How do you think is there any difference between these two approaches?
The answer is yes - difference is quite noticeable... in terms of performance.

First approach takes about 1 second to complete, while the second - nearly 14 seconds! How can this be you may ask?

The answer is memory layout, caching and thrashing.

In .NET much like in C++ arrays are stored row-wise in contiguous memory. So, if you will access array rows first you will access contiguous memory. That means that the next data item you need will be be in pipeline, cache, RAM, and the next hard drive sector before you need it will be in the cache.

But if you go through columns first then you will be repeatedly reading just one item from each row before reading from the next row. As a result your system's caching mechanism and lookahead will fail to give you recent items, and you will waste a lot of time waiting for RAM, which is about three to five times slower than cache. It can get even worse, you may end up waiting for the hard drive. HDD can be millions of times slower than RAM if accessed in large random jumps.

The moral: the less sequential your data access, the slower your program will run :)

Friday, February 01, 2008

When string.ToLower() is Evil


Did you know how evil string.ToLower() can sometimes be?

Let me explain...

Very often I see code similar to this:

void DoBadAction (string val)
{
if (val.ToLower() == "someValue")
{ //do something
}
}

The code above can lead up to 4 times in performance loss when doing string comparison operations.

Best method to do such kind of case insensitive comparison is using string.Equals(...) method.

void DoGoodAction(string val)
{
if (val.Equals("someValue", StringComparison.OrdinalIgnoreCase))
{ //do something
}
}

Why is it so? The reason lies in the string type peculiarity - it is immutable.

Since it is an immutable - string.ToLower() will always return new string instance. Thus generating extra instance of string on every ToLower() call.

Detailed information about string.Equals with StringComparison enumeration can be found here.
Other performance related tips and tricks can be found here.

Wednesday, January 23, 2008

How to Transfer Fixed Sized Data With Async Sockets

This post was inspired by the discussion in MDSN forums.

Discussion on that forum showed that there is misunderstanding of the principles of data transfer across the network, especially in asynchronous way.

I wrote simple client/server application that demonstrates usage of asynchronous sockets to transfer fixed sized data from client to server.

Usually data exchange between server and client is done in some special way (or special order). Such an order is called communication protocol.
More information on network and communication protocols can be found here and here

The sample here also uses communication protocol. It is very simple, client prefixes the data with 4 bytes that hold data size. Format of the data that will go to the wire can be shown like this [4 bytes - data size][data - data size].

Here comes implementation with short comments. Asynchronous communication via sockets involves methods like BeginReceive/EndReceive - for data receiving and BeginSend/EndSend for sending. The sample demonstrates only how async sockets work and the fact that data is received in the stream like way.
More information about network programming can be found here


Server

At first we define the state that will be passed between async calls. State is defined as following:


///
/// Server state holds current state of the client socket
///

class ServerState
{
public byte[] Buffer = new byte[512]; //buffer for network i/o
public int DataSize = 0; //data size to be received by the server
public bool DataSizeReceived = false; //whether prefix was received
public MemoryStream Data = new MemoryStream(); //place where data is stored
public Socket Client; //client socket
}

Server is listening for the clients using Socket.Accept method.
Here is the server listening loop:

//server listening loop
while (true)
{
Socket client = serverSocket.Accept();
ServerState state = new ServerState();

state.Client = client;
//issue first receive
client.BeginReceive(state.Buffer, 0, state.Buffer.Length, SocketFlags.None,
new AsyncCallback(ServerReadCallback), state);
}

When data comes to the client socket ServerReadCallback is called. There server can read received data and process it.

Here is the implementation of that method:

private void ServerReadCallback(IAsyncResult
{
ServerState state = (ServerState)ar.AsyncState;
Socket client = state.Client;
SocketError socketError;

int dataRead = client.EndReceive(ar, out socketError);
int dataOffset = 0; //to simplify logic

if (socketError != SocketError.Success)
{
client.Close();
return;
}

if (dataRead <= 0)
{ //connection reset
client.Close();
return;
}

if (!state.DataSizeReceived)
{
if (dataRead >= 4)
{ //we received data size prefix
state.DataSize = BitConverter.ToInt32(state.Buffer, 0);
state.DataSizeReceived = true;
dataRead -= 4;
dataOffset += 4;
}
}

if ((state.Data.Length + dataRead) == state.DataSize)
{ //we have all the data
state.Data.Write(state.Buffer, dataOffset, dataRead);

Console.WriteLine("Data received. Size: {0}", state.DataSize);

client.Close();
return;
}
else
{ //there is still data pending, store what we've
//received and issue another BeginReceive
state.Data.Write(state.Buffer, dataOffset, dataRead);

client.BeginReceive(state.Buffer, 0, state.Buffer.Length,
SocketFlags.None, new AsyncCallback(ServerReadCallback), state);
}
}

As you can see server implementation is very simple. It receives data size prefix at first and if it is complete (first 4 bytes are received) then proceeds with data receive.

It is very important to always check the number of received bytes. This number can vary.


Client

Client implementation is pretty straightforward. It also has a state that is passed along async operations.

public class ClientState
{
public Socket Client; //client socket
public byte[] DataToSend; //data to be trasferred
public int DataSent = 0; //data already sent
}

The data being sent is prefixed with its size.

ClientState state = new ClientState();
state.Client = socket;

//add prefix to data
state.DataToSend = new byte[data.Length + 4];
byte[] prefix = BitConverter.GetBytes(data.Length);
//copy data size prefix
Buffer.BlockCopy(prefix, 0, state.DataToSend, 0, prefix.Length);
//copy the data
Buffer.BlockCopy(data, 0, state.DataToSend, prefix.Length, data.Length);

socket.BeginSend(state.DataToSend, 0, state.DataToSend.Length,
SocketFlags.None, new AsyncCallback(ClientSendCallback), state);

And finally the implementation of ClientSendCallback

private void ClientSendCallback(IAsyncResult ar)
{
ClientState state = (ClientState)ar.AsyncState;
SocketError socketError;
int sentData = state.Client.EndSend(ar, out socketError);

if ( socketError != SocketError.Success )
{
state.Client.Close();
return;
}

state.DataSent += sentData;

if (state.DataSent != state.DataToSend.Length)
{ //not all data was sent
state.Client.BeginSend(state.DataToSend, state.DataSent,
state.DataToSend.Length - state.DataSent, SocketFlags.None,
new AsyncCallback(ClientSendCallback), state);
}
else
{ //all data was sent
Console.WriteLine("All data was sent. Size: {0}",
state.DataToSend.Length);
state.Client.Close();
}
}

As you can see implementation details are very simple.
There are several important things here:
  • Data is received in pieces, you can't predict how much data you will receive during one method call
  • When using async sockets always pass the same state object in the context of the same client

Update
The code sample above illustrates the idea of tranferring size-prefixed data between client and server in the async way. Please, note that only one message per connection is passed from client to server. To add multiple message handling - server code has to be modified according to your custom the protocol.

Saturday, January 19, 2008

Unmanaged Debugging Option Very Useful in Visual Studio .NET


When developing managed code that is interacting with unmanaged world one has to be very careful, especially when working with unmanaged memory.

Here's an example how critical memory bug can be omitted.

I was developing code that used Marshal.AllocHGlobal method to allocate unmanaged memory. Naturally I was freeing that memory after usage.

The code looked like
IntPtr unmanagedMemory = IntPtr.Zero;
try
{
unmanagedMemory = Marshal.AllocaHGlobal(nBytes);
//invoking unmanaged method here via P/Invoke
}
finally
{
Marshal.FreeHGlobal(unmanagedMemory);
}

Everything was okay; application was working perfectly, except occasional crashes with AccessViolationException. Exception of this type can happen when something is writing into wrong memory offset and system reacts by throwing an exception.

Exception wasn't thrown every time application was running and it was hard to detect what was causing it. At the top of callstack was ntdll.dll module.

I decided to turn on unmanaged debugging and see what is happening under the hood in the unmanaged world.
(To turn unmanaged debugging in Visual Studio you need to go to Project Properties -> Debug -> Check "enable unmanaged code debugging").

After turning on unmanaged debugging I've got an exception immediately.


Marshal.FreeHGlobal(unmanagedMemory) was throwing exception saying that there was heap corruption. This exception was repeating constantly.

Finally, the bug was found - unmanaged function was not behaving well with the pointer passed to it.

Every time you're developing managed code that is interacting with unmanaged memory it is highly desirable to turn on unmanaged debugging. This will save a lot of time when tracking memory related issues and exceptions. Also a lot of information about debugging .NET applications can be found in .NET Debugging

Monday, December 17, 2007

Effective SQL Mindset: Cursors VS Joins



In the past I've dealt a lot with cursors. My experience was mostly negative. SQL code written using them was terribly slow. Thus our team was transforming SQL code with cursors into code without them.

Sure, cursors have their advantages. But, after conversion SQL query performance can be much better then with cursors.

Now, I'll show you how to accomplish the task using SQL with and without cursors.
There are two tables: regions and values. For example, regions represents departments while values is used to store arbitrary int data about the specific region.

SQL tables:
create table regions (id int, name varchar(10))
create table [values] (id int identity(1, 1), regid int, value int)


Table regions contains records for 100 regions, while values contains data only for region with id = 1 (only for the first region). We need to populate values table using data from the first region.

For exaple if table regions contains data





idname
1Reg 1
2Reg 2

And table values contains data only for first region




idReg Idvalue
1130
2135


So we need to populate table values for every region from regions table.

There are three ways how to do this:
  • write a computer program that will insert values into values table
  • write SQL query with cursors
  • write SQL query without cursors
I will not touch how to write a program to do this, its too ineffective.

But last two options seem to be interesting. Cursor solution is quick and dirty - you can code it almost in no time. Cursor code will be similar to the computer program's one.

select id,regionid,value into #Valuestmp from [values]

DECLARE RegionsCursor CURSOR FOR SELECT id FROM regions
declare @id int

OPEN RegionsCursor;
FETCH NEXT FROM RegionsCursor into @id;
WHILE @@FETCH_STATUS = 0
BEGIN

UPDATE #Valuestmp SET RegionId=@id
INSERT INTO [values](regionId, value) SELECT regionId, value FROM #Valuestmp
FETCH NEXT FROM RegionsCursor into @id;
END;
CLOSE RegionsCursor;
DEALLOCATE RegionsCursor;

drop table #Valuestmp
Here we store initial values from values table, and then for each region we insert data into values table.

This elegant script will accomplish the above:
insert into val (regid, value) select r.id as regid, v.value as value from reg r, val v where r.id <> 1

Central point in this script is Cartesian product - we select from two tables (bold font), joining every row of the values table with the rows in regions table.

SQL code above shows how cursor can be avoided. Every time you're tempted to use cursor - stop and think that nearly everything can be done without them. All you have to do is to think little bit.

Thursday, December 06, 2007

Here Comes Another Bubble

Some people say that there won't be .com like bubble any more. However, there are signs out there that indicate the opposite :)



Thanks Alena C++ for the link

Sunday, November 11, 2007

Did you know that...


This post will start the series of "did you know that..." posts. Where I'll try to show things that may be useful for developers and other people connected with IT.

Did you know that "Internal .Net Framework Data Provider error 6" can happen when you use the SQL Native Client data provider to connect to an instance of SQL Server 2005 that is configured to use database mirroring. The solution to this situation is here.

or...

Did you know that "the CPU on a computer that is running the NET Framework 2.0 consumes excessive system resources when you marshal an object array across a domain boundary"?
The solution provided doesn't result in installing hotfixes. But it can force the developer to redesign the app or sacrifice its performance. Both variants are bad :(.

Wednesday, October 24, 2007

Microsoft Announces Mobile Device Manager 2008

Recently Microsoft announced Mobile Device Manager 2008

What they are saying about this new technology:
"Mobile Device Manager 2008 is a solution that gives organizations enhanced on-device security and over-the-air policy enforcement. It allows IT professionals to more easily manage phones within the organization, and gives mobile employees access to confidential information on corporate networks with firewalls."

That means mobile devices can be integrated into corporate IT infrastructure and become part of it. IT personal will be able to control mobile devices and enforce corporate security policies on them.

At least one software company announced support for the Mobile Device Manager 2008.

How this new technology can help software development?

Generally device in the field is connecting to the enterprise network through the Internet. In this case device is initiating connection and is playing the role of client.

With this new technology it is believed that devices will become part of the enterprise network and internal applications will be able to "talk" to devices via secure channel transparently.

Thursday, October 18, 2007

Strategy Pattern in C# 2.0

Strategy pattern is very convenient when it comes to algorithms selection. The selection is possible because algorithm is encapsulated in single class, so it is easier to handle multiple algorithms.

There is a C# sample of Strategy pattern on the Wikipedia. Strategies there are passed as objects into the context. With the emergence of generics in the .NET 2.0 it is possible to specify strategy as a type parameter.

In this post I'll show sample list class "SmartList" that can be tuned with different allocation strategies. Such list can be used when developer needs full control on how memory is consumed by the list. For simplicity, there is no error handling and list class has limited functionality.

So, at first we define our algorithm's interface or contract.


interface IAllocationSrategy<T>
{
T[] Allocate(T[] data)
;
}


Then we define two allocation strategies: DoubleAllocator and FixedAllocator.
DoubleAllocator when doing allocation doubles the initial memory count, while FixedAllocator makes constant increment.


class DoubleAllocator<T> : IAllocationSrategy<T>
{
#region IAllocationSrategy Members

public T[] Allocate(T[] data)
{
int length = data.Length * 2;
T[] newData = new T[length];
Buffer.BlockCopy(data, 0, newData, 0, data.Length);
return
newData;
}

#endregion
}

class FixedAllocator<T> : IAllocationSrategy<T>
{
int fixedIncrement = 20;
#region
IAllocationSrategy Members

public T[] Allocate(T[] data)
{
int length = data.Length + fixedIncrement;
T[] newData = new T[length];
Buffer.BlockCopy(data, 0, newData, 0, data.Length);
return
newData;
}

#endregion
}

SmartList provides list functionality. Items can be added to the list and obtained by index.


class SmartList<A, T> where A : IAllocationSrategy<T>, new()
{
A allocator
= new A();

T[] innerList;
int
size = 0;

public
SmartList() : this(1)
{ }

public SmartList(int? capacity)
{
innerList
= new T[capacity ?? 1];
}

public void Add(T item)
{
if (size == innerList.Length)
innerList
= allocator.Allocate(innerList);
innerList[size++] = item;
}

public T this[int index]
{
get { return innerList[index]; }
}

public int Size
{
get { return size; }
}

public int Capacity
{
get { return innerList.Length; }
}
}



And finally, sample code that uses SmartList class with FixedAllocator and int type as item type.


class Sample
{
public Sample()
{
SmartList<FixedAllocator<
int>, int> smartList =
new
SmartList<FixedAllocator<int>, int>();

smartList.Add(1);
smartList.Add(2);
smartList.Add(3);

Console.WriteLine(smartList.Capacity);
}
}

Thursday, October 04, 2007

String Splitting and Tokenization Techniques in .NET

At start I'll give the explanation what token and tokenization is.

According to wikipedia article: "A token is a categorized block of text, usually consisting of indivisible characters known as lexemes. A lexical analyzer processes lexemes to categorize them according to function, giving them meaning. This assignment of meaning is known as tokenization. A token can look like anything: English, gibberish symbols, anything; it just needs to be a useful part of the structured text."

In .NET string class has a Split method. It returns an array of strings that are substrings delimited by a specified separator. Split method is convenient, however somewhat inefficient. Let us consider an example.

Input string is "59;21;67;72;111". String.Split(';') will return an array of five strings. But what will happen if we need to find specific token, say "21". String.Split will return an array, and then we'll have to iterate through it and search for "21". Do you notice pitfall here? No?

String.Spilt inefficiency comes from its API contract - it returns an array of string, every item of which consumes memory. But we're searching only for "21", we do not need other tokens.

This example shows inappropriate scenario for String.Split(...) method.

Other approach for string tokenizing is using special tokenizers that are returning one token at a time. An example of such tokenizer can be StringTokenizer class from Java world.
Sample code in Java:

StringTokenizer st = new StringTokenizer("this is a test");
while (st.hasMoreTokens()) {
System.out.println(st.nextToken());
}

We can see that StringTokenizer is returning one token at a time. It is extremely efficient for token search scenarios as we may not need to iterate through all the tokens thus allocate less memory and perform faster.

In .NET base class library (BCL) there is no such class. But there are 3-d party analogs or ports of Java StringTokenizer.

.NET BCL actually, has very powerful set of classes that can cope with string tokenization scenario easily. These classes "live" in System.Text.RegularExpressions.
Good example of string tokenization using Regex class is provided by TrackerRealm design team in their blog.

So, there are at least three ways in .NET how strings can be split into tokens. There is a rule of thumb here, if algorithm doesn't need all tokens from the string - do not use String.Split.