Tag Archive: Object Oriented Programming

Aug 29

Fortran dll’s and libraries: a Progress bar

In the previous fortran tutorials, we learned the initial aspects of object oriented programming (OOP) in fortran 2003. And even though our agent-based opinion-dynamics-code is rather simple, it can quickly take several minutes for a single run of the program to finish. Two tools which quickly become of interest for codes that need more than a few minutes to run are: (1) a progress bar, to track the advance of the “slow” part of the code and prevent you from killing the program 5 seconds before it is to finish, and (2) a timer, allowing you to calculate the time needed to complete certain sections of code, and possibly make predictions of the expected total time of execution.

In this tutorial, we will focus on the progress bar. Since our (hypothetical) code is intended to run on High-Performance Computing (HPC) systems and is written in the fortran language, there generally is no (or no easy) access to GUI’s. So we need our progress bar class to run in a command line user interface. Furthermore, because it is such a widely useful tool we want to build it into a (shared) library (or dll in windows).progress_1pct

The progress bar class

What do we want out of our progress bar? It needs to be easy to use, flexible and smart enough to work nicely even for a lazy user. The output it should provide is formatted as follows: <string> <% progress> <text progress bar>, where the string is a custom character string provided by the user, while ‘%progress’ and ‘text progress bar’ both show the progress. The first shows the progress as an updating number (fine grained), while the second shows it visually as a growing bar (coarse grained).

  1. type, public :: TProgressBar
  2. private
  3. logical :: init
  4. logical :: running
  5. logical :: done
  6. character(len=255) :: message
  7. character(len=30) :: progressString
  8. character(len=20) :: bar
  9. real :: progress
  10. contains
  11. private
  12. procedure,pass(this),public :: initialize
  13. procedure,pass(this),public :: reset
  14. procedure,pass(this),public :: run
  15. procedure,pass(this),private:: printbar
  16. procedure,pass(this),private:: updateBar
  17. end type TProgressBar

All properties of the class are private (data hiding), and only 3 procedures are available to the user: initialize, run and reset. The procedures, printbar and updatebar are private, because we intend the class to be smart enough to decide if a new print and/or update is required. The reset procedure is intended to reset all properties of the class. Although one might consider to make this procedure private as well, it may be useful to allow the user to reset a progress bar in mid progress.(The same goes for the initialize procedure.)

  1. subroutine run(this,pct,Ix,msg)
  2. class(TProgressBar) :: this
  3. real::pct
  4. integer, intent(in), optional :: Ix
  5. character(len=*),intent(in),optional :: msg
  6.  
  7. if (.not. this%init) call this%initialize(msg)
  8. if (.not. this%done) then
  9. this%running=.true.
  10. this%progress=pct
  11. call this%updateBar(Ix)
  12. call this%printbar()
  13. if (abs(pct-100.0)<1.0E-6) then
  14. this%done=.true.
  15. write(*,'(A6)') "] done"
  16. end if
  17. end if
  18.  
  19. end subroutine run

In practice, the run procedure is the heart of the class, and the only procedure needed in most applications. It takes 3 parameters: The progress (pct), the number of digits to print of pct (Ix),and the <string> message (msg). The later two parameters are even optional, since msg may already have been provided if the initialize procedure was called by the user. If the class was not yet initialized it will be done at the start of the procedure. And while the progress bar has not yet reached 100% (within 1 millionth of a %) updates and prints of the bar are performed. Using a set of Boolean properties (init, running, done), the class keeps track of its status. The update and print procedures just do this: update the progress bar data and print the progress bar. To print the progress bar time and time again on the same line, we need to make use of the carriage return character (character 13 of the ASCII table):

write(*,trim(fm), advance='NO') achar(13), trim(this%message),trim(adjustl(this%progressString)),'%','[',trim(adjustl(this%bar))

The advance=’NO‘ option prevents the write statement to move to the next line. This can sometimes have the unwanted side-effect that the write statement above does not appear on the screen. To force this, we can use the fortran 2003 statement flush(OUTPUT_UNIT), where “OUTPUT_UNIT” is a constant defined in the intrinsic fortran 2003 module iso_fortran_env. For older versions of fortran, several compilers provided a (non standard) flush subroutine that could be called to perform the same action. As such, we now have our class ready to be used. The only thing left to do is to turn it into a dll or shared library.progress_25pct

How to create a library and use it

There are two types of libraries: static and dynamic.

Static libraries are used to provide access to functions/subroutines at compile time to the library user. These functions/subroutines are then included in the executable that is being build. In linux environments these will have the extension “.a”, with the .a referring to archive. In a windows environment the extension is “.lib”, for library.

Dynamic libraries are used to provide access to functions/subroutines at run time. In contrast to static libraries, the functions are not included in the executable, making it smaller in size. In linux environments these will have the extension “.so”, with the .so referring to shared object. In a windows environment the extension is “.dll”, for dynamically linked library.

In contrast to C/C++, there is relatively little information to be found on the implementation and use of libraries in fortran. This may be the reason why many available fortran-“libraries” are not really libraries, in the sense meant here. Instead they are just one or more files of fortran code shared by their author(s), and there is nothing wrong with that. These files can then be compiled and used as any other module file.

So how do we create a library from our Progressbar class? Standard examples start from a set of procedures one wants to put in a library. These procedures are put into a .f or .f90 file. Although they are not put into a module (probably due to the idea of having compatibility with fortran 77) which is required for our class, this is not really an issue. The same goes for the .f03 or .f2003 extension for our file containing a fortran 2003 class. To have access to our class and its procedures in our test program, we just need to add the use progressbarsmodule clause. This is because our procedures and class are incorporated in a module (in contrast to the standard examples). Some of the examples I found online also include compiler dependent pragmas to export and import procedures from a dll. Since I am using gfortran+CB for development, and ifort for creating production code, I prefer to avoid such approaches since it hampers workflow and introduces another possible source of bugs.

The compiler setups I present below should not be considered perfect, exhaustive or fool-proof, they are just the ones that work fine for me. I am, however, always very interested in hearing other approaches and fixes in the comments.progress_52pct

Windows

The windows approach is very easy. We let Code::Blocks do all the hard work.

shared library: PBar.dll

Creating the dll : Start a new project, and select the option “Fortran DLL“. Follow the instructions, which are similar to the setup of a standard fortran executable. Modify/replace/add the fortran source you wish to include into your library and build your code (you can not run it since it is a library).

Creating a user program : The program in which you will be using the dll is setup in the usual way. And to get the compilation running smoothly the following steps are required:

  • Add the use myspecificdllmodule clause where needed, with myspecificdllmodule the name of the module included in the dll you wish to use at that specific point.
  • If there are modules included in the dll, the *.mod files need to be present for the compiler to access upon compilation of the user program. (Which results in a limitation with regard to distribution of the dll.)
  • Add the library to the linker settings of the program (project>build options>linker settings), and then add the .dll file.
  • Upon running the program you only need the program executable and the dll.

static library

The entire setup is the same as for the shared library. This time, however, choose the “Fortran Library” option instead of Fortran dll. As the static library is included in the executable, there is no need to ship it with the executable, as is the case for the dll.

Unix

For the unix approach we will be working on the command line, using the intel compiler, since this compiler is often installed at HPC infrastructures.

static library: PBar.a

After having created the appropriate fortran files you wish to include in your library (in our example this is always a single file: PBar.f03, but for multiple files you just need to replace PBar.f03 with the list of files of interest.)

  1. Create the object files:
    ifort -fpic -c -free -Tf Pbar.f03

    Where -fpic tells the compiler to generate position independent code, typical for use in a shared object/library, while -c tells the compiler to create an object file. The -free and -Tf compiler options are there to convince the compiler that the f03 file is actual fortran code to compile and that it is free format.

  2. Use the GNU ar tool to combine the object files into a library:
    ar rc PBarlib.a PBar.o
  3. Compile the program with the library
    ifort TestProgram.f90 PBarlib.a -o TestProgram.exe

    Note that also here the .mod file of our Progressbarsmodule needs to be present for the compilation to be successful.

shared library: PBar.so

For the shared library the approach does not differ that much.

  1. Create the object files:
    ifort -fpic -c -free -Tf Pbar.f03

    In this case the fpic option is not optional in contrast to the static library above. The other options are the same as above.

  2. Compile the object files into a shared library:
    ifort -shared PBar.o -o libPBar.so

    The compiler option -shared creates a shared library, while the -o option allows us to set the name of the library.

  3. Compile the program with the library
    ifort TestProgram.f90 libPBar.so -o TestProgram.exe

    Note that also here the .mod file of our Progressbarsmodule needs to be present for the compilation to be successful. To run the program you also need to add the location of the library file libPBar.so to the environment variable LD_LIBRARY_PATH

One small pickle

HPC systems may perform extensive buffering of data before output, to increase the efficiency of the machine (disk-writes are the slowest memory access option)…and as a result this can sometimes overrule our flush command. The progressbar in turn will not show much progress until it is actually finished, at which point the entire bar will be shown at once. There are options to force the infrastructure not to use this buffering (and the system administrators in general will not appreciate this), for example by setting the compiler flag -assume nobuffered_stdout. So the best solution for HPC applications will be the construction of a slightly modified progress bar, where the carriage return is not used.

progress_100pct

 

Special thanks also to the people of stack-exchange for clarifying some of the issues with the modules.

Source files for the class and test-program can be downloaded here.

 

May 22

Tutorial OOP(II): One problem, different possible classes

additional resources
agent paper: Sobkowicz
source-code: AgentTutorials
Arxiv: Full Tutorial

In the previous tutorial, we saw how to tackle an opinion dynamics problem using agents as a class in an Object Oriented Programming (OOP) approach. In many topics of interest in (socio-)physics and chemistry, we deal with a large number of particles, be it electrons, atoms, agents, stars, … These are contained in a superstructure (electrons⇒atom, atoms⇒molecule/solid, agents⇒population, stars⇒galaxy,…) which is generally represented in the code as an array. As we noted in the previous tutorial, there were several variables which were global to the agents, but we implemented them as properties of the agents anyhow. As a result, a significant amount of additional memory needed to be allocated for storing in essence the same data. This was done to prevent the need of having to provide this information at every function call.

Returning to our problem of interest, we now consider two object classes: The TAgent-class and the TPopulation-class. This leads to several possible ways this problem can be implemented.

  1. Array of TAgents: As was done in the previous tutorial, we only make a class of the agents, and put them in an array.
  2. TPopulation of TAgents: In this case we construct a class called TPopulation of which one property is the set of TAgents. The TPopulation-class also contains some of the global variables as properties, and operations on this set are methods of the TPopulation-class.
  3. TPopulation-class without TAgent-class: In this last case, the agents are dissolved, and their properties are stored in array-properties of the TPopulation-class. The methods of the TAgent-class now become methods of the TPopulation-class. And the global variables become additional properties of the TPopulation-class.

Although the true OOP-programmer may only consider the  second option the way to go, we will consider the third option in this tutorial.

Read the rest of this entry »

May 03

Tutorial OOP(I): Objects in Fortran 2003

After having set up our new project in the first session of this tutorial, we now come to an important second step: choosing and creating our Objects. In OOP, the central focus is not a (primitive) variable or a function, but “an object”. In Object Oriented Programming (OOP) most if not all variables and functions are incorporated in one or more (types of) objects. An object, is just like a real-life object; It has properties and can do things. For example: a car. It has properties (color, automatic or stick, weight, number of seats,…) and can do things (drive, break down, accelerate,…). In OOP, the variables containing the values that give the color, weight, stick or not,… of the car are called the properties of the car-object. The functions that perform the necessary calculations/modifications of the variables to perform the actions of driving, breaking down,… are called the methods.

Since we are still focusing on the opinion dynamics paper of Sobkowicz, let us use the objects of that paper to continue our tutorial. A simplified version of the research question in the paper could be as follows:

How does the (average) opinion of a population of agents evolve over time?

For our object-based approach, this already contains much of the information we need. It tells us what our “objects” could be: agents. It gives us properties for these objects: opinion. And it also tells us something of the methods that will be involved: opinion…evolve over time.

Let us now put this into Fortran code. A class definition in Fortran uses the TYPE keyword, just like complex data types.

  1. Type, public :: TAgentClass
  2. private
  3. real :: oi !< opinion
  4. contains
  5. private
  6. procedure, pass(this), public :: getOpinion
  7. procedure, pass(this), public :: setOpinion
  8. procedure, pass(this), public :: updateOpinion
  9. end type TAgentClass

Read the rest of this entry »

Apr 26

Tutorial: Starting a new Object Oriented Fortran-project in Code::Blocks

Because Fortran has been around since the time of the dinosaurs, or so it seems, one persistent myth is that modern constructs and techniques are not supported. One very important modern programming approach is Object Oriented Programming (OOP). Where procedural programming focuses on splitting the program in smaller functional units (functions and procedures), OOP is centered on “Objects and Classes”. These objects, just like real life objects, have properties (eg. name, color, position, …) and contain methods (i.e. procedures representing their functionality). This sounds more complex than it actually is. In a very simplistic sense, objects and classes are created by extending complex data types in procedural programming languages with internally contained functions/procedures, referred to as methods. In the table below, you see a limited comparison between the two. The first three languages are procedural oriented languages, while the last two allow for OOP.

C Pascal F95 C++ OOP F2003 OOP
data type keyword struct record type class type
containing variables field field field property property
containing functions / / / method method
inheritance no no no yes yes

In this tutorial series, I want to show how you can use OOP in Fortran in scientific codes. As a first example, I will show you how to create an agent based opinion dynamics code implementing the model presented by Sobkowicz in a publication I recently had  to review.

I choose this as our tutorial example (over for example geometric shapes) because it is a real-life scientific example which is simple enough to allow for several aspects in scientific and OO-programming to be shown. In addition, from the physicists point of view: particle or agent, it is all the same to us when we are implementing their behavior into a program.(You could even start thinking in terms of NPC’s for a game, although game-dynamics in general require quite a bit more implementation)

So let us get started. First, you might wish to grab a copy of the paper by Sobkowicz (it’s freely accessible), just to be able to more easily track the notation I will be using in the code. After having installed code::blocks it is rather straight-forward to create a new Fortran program.

  1. Open code::blocks, and start a new project (File > New > Project) by selecting “Fortran application”newProject
  2. Provide a “Project title” and a location to store the files of the new project.setupProject1
  3. Choose your compiler (GNU Fortran Compiler, make sure this is the one selected) and Finish.setupProject2
  4. Now Code::Blocks will present you a new project, with already one file present: main.f90 HelloWorld
  5. First we are going to rename this main file, e.g. to main.f95, to indicate that we will be  using Fortran 95 code in this file. By right-clicking on the file name (indicated with the red arrow) you see a list of possible op w this option while the file is open in the right-hand pane. Close the file there, and now you can rename main.f95.
  6. Although we will be following the OOP paradigm for the main internal workings of our program, a procedural setup will be used to set up the global lines of the program. I split the code in three levels: The top-level being the main program loop which is limited to the code below, the middle level which contains what you may normally consider the program layout (which you can easily merge with the top level) and the bottom level which contains the OO innards of our program. All subroutines/functions belonging to the middle level will be placed in one external module (Tutorial1) which is linked to the main program through the use-statement at line 2 and the single subroutine call (RunTutorial1).
  1. program AgentTutorials
  2. use Tutorial1;
  3. implicit none
  4.  
  5. write(*,'(A)') "Welcome to the Agents OOP-Fortran Tutorial"
  6. call RunTutorial1()
  7. write(*,'(A)') "-- Goodbye"
  8. end program AgentTutorials
  1. Next step, we need a place for the RunTutorial1 subroutine, and all other subroutines/functions that will provide the global layout of our agents-program. Make a new file in Code::Blocks as followsNewFile
  2. and select a Fortran source file and finish by placing it in a directory of your choice and click OK.NewFile2
  3. In this empty file, a module is setup in the following fashion:
  1. module Tutorial1
  2. implicit none
  3. private
  4.  
  5. public :: RunTutorial1
  6.  
  7. contains
  8. !+++++++++++++++++++++++++++++++++++++
  9. !>\brief Main program routine for the first agents tutorial
  10. !!
  11. !! This is the only function accessable outside this module.
  12. !<------------------------------------
  13. subroutine RunTutorial1()
  14. write(*,'(A)') "----Starting Turorial 1 subprogram.----"
  15. write(*,'(A)') "-------End Turorial 1 subprogram.------"
  16. end subroutine RunTutorial1
  17.  
  18. end module Tutorial1
  1. implicit none : Makes sure that you have to define all variables.
  2. private : The private statement on line 3 makes sure that everything inside the module remains hidden for the outside world, unless you explicitly make it visible. In addition to data/information-hiding this also helps you keep your namespace in check.
  3. public :: Runtutorial1 : Only the function Runtutorial1 is available outside this module.(Because we want to use it in program  AgentTutorials.)
  4. contains : Below this  keyword all functions and subroutines of the module will be placed.
  5. “!” : The exclamation marks indicate a comment block. In this case it is formatted to be parsed by doxygen.
  6. subroutine XXX  / end subroutine XXX : The subroutine we will be implementing/using as the main loop of our agents program. Note that in recent incarnations of fortran the name of the subroutine/function/module/… is repeated at their end statement. For small functions this may look ridiculous, but for larger subroutines it improves readability.

At this point or program doesn’t do that much, we have only set up the framework in which we will be working. Compiling and running the program (F9 or Build > Build and run) should show you:

AgentsRun1 In the next session of this tutorial we will use OO-Fortran 2003 to create the agent-class and create a first small simulation.

 

 

Jan 16

CA: Coders Anonymous

It has been 2 weeks since I last wrote some code, today I started again.

Two weeks ago  I started the, for me, daunting task of upgrading my IDE and compiler to their most recent version. The upgrade itself went smoothly, since it basically consisted of uninstalling the old versions and installing the new ones. The the big finale, recompiling my fortran codebase, went just a little bit less smoothly. It crashed straight into a compiler-bug, nicely introduced in version 4.9 of the gcc fortran compiler, and carefully nurtured up to version 4.10 I had just installed. The bug sounds as follows:

error: internal compiler error: in gfc_conv_descriptor_data_get, at fortran/trans-array.c:145
end module TPeriodicTableModule

Clear sounding as it is, it required some further investigation to find out what was actually the problem and if and how it can be resolved. The problem appeared to be a rather simple one; The compiler seems to be unable to generate the finalization code for some object based constructions involving both fixed size and allocatable arrays, the strong suite of the fortran language.  A minimal example allowing you to bump into this compiler bug goes as follows:

 

Bug 59765   
  1. module bug59765
  2. type TSubObject
  3. integer, dimension(:), allocatable :: c
  4. end type TSubObject
  5. type TObject
  6. type(TSubObject), dimension(1) :: u
  7. end type TObject
  8. contains
  9.  
  10. subroutine add(s)
  11. class(TObject), intent(inout) :: s
  12. end subroutine add
  13.  
  14. end module bug59765

 

The issue arises when the compiler tries to setup the deallocation of the allocatable array of the TSubObject elements of the array u. Apparently the combination of the static array u and allocatable arrays c in the elements of u result in confusion. It was suggested that the compiler wants to perform the deallocation procedure as an array operation (one of the neat tricks fortran has up its sleeves):

deallocate(s%u(:)%c)

instead it should just use a normal do-loop and run over all elements of u.

One of the main ironies of this story is that this bug is strongly connected to object oriented programming, a rather new concept in the world of fortran. Although introduced in fortran 2003, more than 10 year ago, compiler support for these features have only reached basic maturity in recent years. The problem we are facing is one in the destructor of an object: the smart compiler wants to make our life easy, and implicitly create a good destructor for us. As with most smart solutions of modern day life, such things have a tendency to fail when you least expect it.

However, this bug (and the fact that it persists in more recent versions of the compiler) forces us to employ good coding practices: write a destructor yourself. Where C++ has implemented keywords for both constructor and destructor, the fortran programmer, as yet, only has a keyword for a destructor: final. This finalization concept was introduced in the fortran 2003 standard as part of the introduction of the Object Oriented Programming paradigm. A final procedure/function also works slightly different than what you may be used to in for example C++, namely, it is not directly callable by the programmer as an object-function/procedure. A final procedure/function is only called upon in an automatic way when an object is destroyed. So for those of us who also implement  ‘free()‘ procedures to clean-up objects at runtime, this means some extra work may be needed (I haven’t checked this in detail).

So how is our example-problem healed from bug 59765? Through the introduction of our own destructor.

  1. module fixbug59765
  2. type TSubObject
  3. integer, dimension(:), allocatable :: c
  4. contains
  5. final :: destroy_TSubObject
  6. end type TSubObject
  7. type TObject
  8. type(TSubObject), dimension(1) :: u
  9. end type TObject
  10. contains
  11.  
  12. subroutine add(s)
  13. class(TObject), intent(inout) :: s
  14. end subroutine add
  15.  
  16. subroutine destroy_TSubObject(this)
  17. type(TSubObject) :: this !note: this needs to be a type not a class
  18.  
  19. if (allocated(this%c)) deallocate(this%c)
  20. end subroutine destroy_TSubObject
  21.  
  22. end module fixbug59765

In my own code, both the TSubObject and TObject classes got their own final procedure, due to the slightly higher complexity of the objects involved. The resulting code compiled without further complaints, and what is more, it also still compiled with the recent intel ifort compiler. Unfortunately, final procedures are only included in the gcc compiler since version 4.9, making code containing them incompatible with the gcc version 4.8 and earlier.