Experimental Project Report The Interaction Between Windows Operating System and Microsoft .NET Framework

Raymond Tan

Contents Contents ............................................................................................................................................ i Abstract ........................................................................................................................................... ii 1 Experiment One - Multithreading ................................................................................................ 1 1.1 Introduction ........................................................................................................................... 1 1.1.1 Purpose ........................................................................................................................... 1 1.2 Threading in C# and VB.NET ............................................................................................... 1 1.3 Synchronisation ..................................................................................................................... 2 1.4 BackgroundWorker Class ...................................................................................................... 2 1.5 Summary ............................................................................................................................... 3 2 Experiment Two - Memory Management .................................................................................... 4 2.1 Introduction ........................................................................................................................... 4 2.1.1 Purpose ........................................................................................................................... 4 2.2 Dynamic Memory Allocation ................................................................................................ 4 2.3 Automatic Garbage Collection .............................................................................................. 5 2.4 Summary ............................................................................................................................... 7 3 Experiment Three - File System ................................................................................................... 7 3.1 Introduction ........................................................................................................................... 7 3.1.1 Purpose ........................................................................................................................... 8 3.2 Assumption............................................................................................................................ 8 3.3 Design.................................................................................................................................... 8 3.4 Implementation .................................................................................................................... 10 3.5 Summary ............................................................................................................................. 10 4 Appendices ................................................................................................................................. 11 4.1 Appendix A: Application Screenshots ................................................................................ 11 4.1.1 Matrix calculation using a BackgroundWorker ............................................................ 11 4.1.2 File Divider and Combiner ........................................................................................... 12 4.2 Appendix B: References ...................................................................................................... 13

i

Abstract This project consists of three experiments on three different aspects of interaction between Windows operating system and Microsoft .NET Framework. That is, the main focus is on parallel programming, memory management and file system. The first experiment is to discuss parallel programming with C# and VB.NET and create applications for a demonstration of multithreading. The second experiment is to explore the techniques and mechanisms for avoiding unnecessary memory long-tern fragmentation. The last experiment is about how Microsoft .NET Framework handles file operations and is expected to resolve a practical storage problem. In order to carry out the experiments, I conducted an in-depth research on Microsoft .NET Framework. From Microsoft Developer Network (MSDN), I collected an amount of detailed information on technologies applied to communication between Windows operating system and Microsoft .NET Framework. The result of the experiments shows that Microsoft .NET Framework provides many effective means of using available resource from the operating system. Because of that, it is very useful for developers to create robust applications.

Keywords: Multithreading, Memory Management, File System, Application, .NET Framework

ii

1 Experiment One - Multithreading 1.1 Introduction Most computers nowadays have multiple CPUs which allow a computer to run more than one program simultaneously by hardware support (Michaelis, 2010). It can be more effective if a program is able to execute more than one sequence of instructions. And multithreading is the solution to this problem.

1.1.1 Purpose The main purpose of this experiment is to briefly discuss C# and VB.NET multithreaded applications and demonstrate some examples (Appendix A: 4.1.1 Matrix calculation using a BackgroundWorker). The thread synchronisation and the special class BackgroundWorker are the major discussion.

1.2 Threading in C# and VB.NET A C# or Visual Basic application is single-threaded by default. Parallel programming with C# and Visual Basic enables an application to have auxiliary threads to perform different tasks simultaneously. In a desktop application, for instance, auxiliary threads are able to execute timeconsuming tasks in the background (MSDN, 2011). One of the advantages of using auxiliary threads in a desktop application is that the primary thread handling the GUI of the application is not interrupted by performing the time-consuming tasks. The following diagram illustrates the structure of the parallel programming in the newest version of Microsoft .NET Framework.

(Source: Parallel Programming in Microsoft .NET Framework) Figure 1: The overview of the parallel programming architecture 1

1.3 Synchronisation There are several means of performing thread synchronisation in a C# or Visual Basic application. Using the lock (C#) and SyncLock (Visual Basic) statements is the effective way to create a critical section for synchronisation (MSDN, 2011). Those statements provide a mutual exclusion lock for preventing a concurrent execution in the same code block. Another way to establish a block of code is to use the static methods Enter and Exit from the Monitor class (MSDN, 2011). Both ways to create a thread-safe application are very similar. However, “Using the lock (C#) or SyncLock (Visual Basic) keyword is generally preferred over using the Monitor class directly, both because lock or SyncLock is more concise, and because lock or SyncLock insures that the underlying monitor is released, even if the protected code throws an exception.” (MSDN, 2011) The equivalent code as shown in the following table demonstrates how to use the lock (C#) statement and the Monitor class. lock statement

Monitor class

List list = new List(); lock (list) { list.Add(100); }

List list = new List(); System.Threading.Monitor.Enter(list); try { list.Add(100); } finally { System.Threading.Monitor.Exit(list); }

NOTE:  The argument for lock statement must be a reference type.  Before using threads in C# and Visual Basic, the namespace System.Threading in the .NET Framework must be imported.

In addition, there is a similar class named Mutex (MSDN, 2011) in the .NET library which is able to create a safe thread synchronisation for applications. For atomic operations, the Interlocked class provides several methods for simultaneous modification of shared values (MSDN, 2011). Despite the fact that there are a few means of thread synchronisation, a thread still cannot interact with others. Therefore, the .NET Framework provides two synchronisation events AutoResetEvent (MSDN, 2011) and ManualResetEvent (MSDN, 2011) for thread interaction. Those events assist a thread in notifying one or more waiting threads when an event arises. To control threads enter a critical section, furthermore, those synchronisation events play a vital role in thread communication.

1.4 BackgroundWorker Class The BackgroundWorker class (MSDN, 2011), as a component class in Microsoft .NET Framework, provides a few methods and events for developers to complete time-consuming tasks such as heavy downloads, massive calculations, database transactions, etc. The essential part of 2

the BackgroundWorker class is the DoWork event. This event is able to perform a task in the background of an application. The trigger of the event is the method RunWorkerAsync. Once that method is called, a thread automatically created by an instance of the BackgroundWorker class starts executing the operation. As mentioned previously, the time-consuming operations may cause a problem, which is the user interface is unresponsive whilst the application is running a heavy task. Using the Background Class, therefore, is one of the solutions to that problem. The following illustration shows the main properties, methods and events in the BackgroundWorker class.

(NOTE: Created in Microsoft Visual Studio 2010) Figure 2: The class diagram of the BackgroundWorker class

In addition, if an instance of the BackgroundWorker class needs to communicate with the user interface, there are another two events for that. For instance, the application needs to report the progress according to the tasks that have been completed. Using the ProgressChanged event can retrieve the data and concurrently show it in the user interface. The method ReportProgress is the provider of the progress information. Another event, RunWorkerCompleted, is to notify the application when the tasks are accomplished (MSDN, 2011).

1.5 Summary As can be seen, Microsoft .NET Framework provides a few useful methods and a convenient component that assist developers creating multithreaded applications. In fact, Microsoft .NET Framework provides some other multithreaded components such as System.Timers.Timer. With all those methods and components, developers are able to build a robust and reliable multithreaded application easily. 3

2 Experiment Two - Memory Management 2.1 Introduction The principle of executing an application in a computer is that the application must be loaded into memory before passing its instructions to CPU for execution. Thus, memory allocation plays a vital role in an operating system. Because of the different size of processes, there may be a fragmentation problem. There are three types of fragmentation: external fragmentation, internal fragmentation and data fragmentation (Wikipedia, 2011). Memory Management in Microsoft .NET Framework is very sophisticated to resolve fragmentation problems.

2.1.1 Purpose The main purpose of this experiment is to briefly discuss how Microsoft .NET Framework manages memory long-term fragmentation. Data fragmentation and garbage collection are the main discussion.

2.2 Dynamic Memory Allocation This chapter intends to elaborate the process of memory allocation in Microsoft .NET Framework. When a .NET application is loaded into memory, common language runtime (CLR), the critical portion of Microsoft .NET Framework, needs to be allocated to a free and contiguous memory to create a managed heap (MSDN, 2000). The managed heap is to store objects created by the application. In the managed heap, there is a pointer to indicate the address of next object to be allocated in. When a manage heap is full, CLR automatically creates another managed heap. By default, two initial managed heaps are established in the memory when an application is initialised (MSDN, 2008). One managed heap is to handle the small objects; the other one is to handle the large objects. If the size of an object is less than 85, 000 bytes (MSDN, 2008) which is approximately 83 KB, the object will be placed in the small object heap (SOH). On the other hand, the object will be put in the large object heap (LOH). Comparing to C language, dynamic memory allocation in C is to use a linked list as a heap to manage all the objects (MSDN, 2000). Thus, the object must check through the entire heap in order to be allocated in the memory (MSDN, 2000). It is more than likely that Microsoft .NET Framework is more effective and simpler to manage memory for applications. The following illustration is to demonstrate memory allocation for .NET applications.

4

(NOTE: Created in Microsoft PowerPoint 2007) Figure 3: Memory allocation in Microsoft .NET Framework

2.3 Automatic Garbage Collection There is a garbage collector in Microsoft .NET Framework to manage memory (MSDN, 2003). Garbage collector provides a very effective memory allocation service, so that the fragmentation problems can be resolved quickly. Garbage collector is able to automatically detect those objects which are no longer used by the application. If those dead objects are found, it will collect them and release the memory (MSDN, 2000). To collect garbage (dead objects) in the managed heap, garbage collector has two ways to fulfil its tasks. One way is to make a complete collection. Firstly, the application must be paused, so that garbage collector can work properly. In fact, this is the safe way for garbage collector to reallocate the objects and reclaim the wasted space (MSDN, 2003). Secondly, the garbage collector starts seeking the root objects in the managed heap. And it continuously checks through the entire managed heap to search those reachable objects. If such objects are found, the garbage collector will rearrange the managed heap. Other unreachable objects are treated as dead objects which are collected automatically (MSDN, 2003). After a complete collection, the application will be automatically resumed. The following diagram shows the process of garbage collection.

5

(NOTE: Created in Microsoft PowerPoint 2007) Figure 4: The process of garbage collection

As can be seen, making a complete garbage collection consumes a considerable amount of time and resources. The alternative way to collect garbage is to make a partial collection in the manage heap. A partial collection is a generational collection (MSDN, 2000). Garbage collector has three generations to collect dead objects. The first generation, Generation 0, is to remove the temporary objects such as an object to connect database, an object to retrieve configurations, etc. (MSDN, 2008) The second generation, Generation 1, is to eliminate those objects not being pointed by the old objects (MSDN, 2008). In fact, this is the way how garbage collector solves the problem of data fragmentation (Wikipedia, 2011). For instance, one of the nodes is removed from a linked list but it still remains in the memory. The garbage collector starts looking for such objects from the root object and disposes of them if they are found. The following illustration shows how the garbage collector reclaims the memory from a linked list.

(NOTE: Created in Microsoft PowerPoint 2007) Figure 5: Garbage collector in generation one 6

After collecting garbage in generation 0 and 1, most dead objects are collected from the managed heap. Generation 2, the last generation, is to collect the large objects. Generation 2 is virtually a complete garbage collection as mentioned previously (MSDN, 2000). Therefore, all the dead objects in the entire managed heap are collected when the last generation collection is completed. In addition, there are several features in Microsoft .NET Framework to support the garbage collector. For example, implementing the IDisposable interface (MSDN, 2011) in C# is a good way to collect a temporary object. In addition, temporary objects also can be disposed by the using statement (MSDN, 2011). The following example shows how to implement the IDisposable interface and how to use the using statement. IDisposable interface

using statement

2.4 Summary Dynamic memory allocation and automatic garbage collection are very effective to avoid memory long-term fragmentation. Microsoft .NET Framework simplifies the complicated process of memory management. Thus, it is very useful for developers to allocate memory resource for applications.

3 Experiment Three - File System 3.1 Introduction Computer has a capacity for storing information on physical devices. That kind of information is treated as files in a computer. The operating system provides file formats which assist users in file management. File formats are defined by the information creators which are applications (Silberschatz, Galvin and Gagne, 2005). Microsoft .NET Framework provides a number of safe functions for communication between applications and the file system in Windows.

7

3.1.1 Purpose This experiment is expected to be more practical. The purpose of this experiment is to create an application (Appendix A: 4.1.2 File Divider and Combiner) which is able to divide a file into several files and combine those files to retrieve the original file. If such application is successfully created, there will be a solution to the problem that a large file cannot be transferred due to lack of space on a storage device. For example, it is unlikely that a file which the size is more than 25 MB can be sent in an email attachment (Wikipedia, 2011). And it is not possible to move a file with 3 GB data from computers to computers by using a USB stick that only has 2 GB free space.

3.2 Assumption According to the Microsoft’s description (MSDN, 211), files are treated as a stream with contiguous data in Microsoft .NET Framework. A stream, which has a beginning and an end, can be read, written and sought (MSDN, 211). Furthermore, the authors (Silberschatz, Galvin and Gagne, 2005) of the textbook stated that “an object file is a sequence of bytes organized into blocks understandable by the system's linker.” Therefore, the assumption of this experiment is that - if a file can be divided into several parts of bytes stored in object files, then it is possible to combine them to retrieve the original file without any flaws. The following diagram shows the overview of the assumption.

(NOTE: Created in Microsoft Office PowerPoint 2007) Figure 6: The overview of the assumption

3.3 Design The application should be designed to perform two main tasks. One task is to divide a selected file into two or more object files, and the other one is to retrieve the original file from all the object files. Moreover, the information of the file properties such as creation time, extension, size, etc. should be provided by the application. The following illustration is an activity diagram demonstrating how the application completes the process of dividing and combining files.

8

(NOTE: Created in Microsoft Office Visio 2007) Figure 7: The activity diagrams of File Divider and Combiner

In the process of dividing a file, the application creates a new file format in order to support the object files. As can be seen, every object file is a part of the original file. Therefore, the File Combiner, which is the linker (Wikipedia, 2011), retrieves the data from all the object files and store them in a new file. By the end of the iterative process, the original data and file format are retrieved. The original file is therefore re-created. Technically, the iterative process is to copy data from object files. Thus, the linker can be a batch file (.bat) with several MS-DOS commands (Wikipedia, 2011) to complete the entire process of file combination. In addition, the application provides a log function that stores the information of the process in a text file. 9

3.4 Implementation In this experiment, Visual Basic is selected as the implementation language. To complete this experiment, the following libraries in Microsoft .NET Framework are applied in the application.   

Threading (System.Threading.Thread) Interaction of File System (System.IO) Memory Garbage Collection (System.GC)

Reading from or writing to a file can be a time-consuming task. Thus, Threading must be applied to the application in order to assist the application in remaining responsive user interface. In Microsoft .NET Framework, the FileStream class provides a number of methods for file operations (MSDN, 2011). If the application attempts to read data from a file or write data to a file, the BinaryReader (MSDN, 2011) and BinaryWriter (MSDN, 2011) classes must be also applied. The instances of those classes in the application are temporary objects. Once those objects accomplish their tasks, they are collected by memory garbage collector. In addition, Microsoft .NET Framework provides the FileInfo (MSDN, 2011) class which is able to get the information of file properties. In the implementation, those classes mentioned above play a crucial role in communicating with the file system in Windows.

3.5 Summary As the expected application is established and tested, the results of the experiment shows that - a file in a Windows operating system can be divide into several object files which can be combined to retrieve the original file successfully. Furthermore, the external linker, the batch file, succeeded in retrieving the original file as well. Hence, the assumption of the experiment is proved by the results. This is, to some extent, a decent solution to the problem that a large file cannot be transferred due to lack of space on a storage device.

10

4 Appendices 4.1 Appendix A: Application Screenshots 4.1.1 Matrix calculation using a BackgroundWorker

Figure 8: Matrix calculation using a BackgroundWorker (C# Example).

Figure 9: Matrix calculation using a BackgroundWorker (VB.NET Example). 11

4.1.2 File Divider and Combiner

Figure 10: File Divider

Figure 11: File Combiner 12

Figure 12: Files created by File Divider

4.2 Appendix B: References [1] Michaelis, M., 2010. Essential C# 4.0. Boston: Pearson Education. [2] MSDN, 2011. Threading (C# and Visual Basic). [online] Available at: [Accessed 07 October 2011]. [3] MSDN, 2011. Parallel Programming in the .NET Framework. [online] Available at: [Accessed 07 October 2011]. [4] MSDN, 2011. Thread Synchronization (C# and Visual Basic). [online] Available at: [Accessed 08 October 2011]. [5] MSDN, 2011. Mutex Class (System.Threading). [online] Available at: [Accessed 09 October 2011]. [6] MSDN, 2011. Interlocked Class (System.Threading). [online] Available at: [Accessed 09 October 2011].

13

[7] MSDN, 2011. AutoResetEvent Class (System.Threading). [online] Available at: [Accessed 09 October 2011]. [8] MSDN, 2011. ManualResetEvent Class (System.Threading). [online] Available at: [Accessed 09 October 2011]. [9] MSDN, 2011. BackgroundWorker Class (System.ComponentModel). [online] Available at: [Accessed 09 October 2011]. [10] Wikipedia, 2011. Fragmentation (computer). [online] Available at: [Accessed 18 October 2011]. [11] MSDN, 2000. Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework. [online] Available at: [Accessed 19 October 2011]. [12] MSDN, 2008. CLR Inside Out: Large Object Heap Uncovered. [online] Available at: [Accessed 19 October 2011]. [13] MSDN, 2003. Garbage Collector Basics and Performance Hints. [online] Available at: [Accessed 21 October 2011]. [14] MSDN, 2011. IDisposable Interface (System). [online] Available at: [Accessed 21 October 2011]. [15] MSDN, 2011. using Statement (C# Reference). [online] Available at: [Accessed 21 October 2011]. [16] Silberschatz, A. Galvin P. and Gagne G., 2005. Operating System Concepts with Java. 7th ed. Hoboken: John Wiley & Sons, Inc. [17] MSDN, 2011. Basics of .NET Framework File I/O and the File System (Visual Basic). [online] Available at: [Accessed 21 October 2011]. [18] Wikipedia, 2011. Object file. [online] Available at: [Accessed 22 October 2011]. [19] Wikipedia, 2011. E-mail attachment. [online] Available at: [Accessed 22 October 2011]. [20] Wikipedia, 2011. Linker (computing). [online] Available at: [Accessed 22 October 2011]. [21] Wikipedia, 2011. List of MS-DOS commands. [online] Available at: [Accessed 23 October 2011]. 14

[22] MSDN, 2011. FileStream Class (System.IO). [online] Available at: [Accessed 23 October 2011]. [23] MSDN, 2011. BinaryReader Class (System.IO). [online] Available at: [Accessed 23 October 2011]. [24] MSDN, 2011. BinaryWriter Class (System.IO). [online] Available at: [Accessed 23 October 2011]. [25] MSDN, 2011. FileInfo Class (System.IO). [online] Available at: [Accessed 24 October 2011].

15