Tag: artificial intelligence

Oct 22 2023

Materiomics Chronicles: week 5

After week four, this fifth week of the academic year is most arguably the most intense and hectic week of teaching. With 22h of classes and still two classes that needed to be prepared from scratch (even including weekends time was running out), I’m tired but happy it is over. However, all the effort is worth it, and I was happy to hear the students of our materiomics program at UHasselt appreciate the effort put into creating their classes, during an evaluation meeting.

The corral is an artificial structure created from 48 iron atoms (the sharp peaks) on a copper surface. The wave patterns in this scanning tunneling microscope image are formed by copper electrons confined by the iron atoms. Don Eigler and colleagues created this structure in 1993 by using the tip of a low-temperature scanning tunneling microscope (STM) to position iron atoms on a copper surface, creating an electron-trapping barrier. This was the first successful attempt at manipulating individual atoms and led to the development of new techniques for nanoscale construction.source: https://www.nisenet.org/catalog/scientific-image-quantum-corral-top-view

The corral is an artificial structure created from 48 iron atoms (the sharp peaks) on a copper surface. The wave patterns in this scanning tunneling microscope image are formed by copper electrons confined by the iron atoms. Don Eigler and colleagues created this structure in 1993 by using the tip of a low-temperature scanning tunneling microscope (STM) to position iron atoms on a copper surface, creating an electron-trapping barrier. This was the first successful attempt at manipulating individual atoms and led to the development of new techniques for nanoscale construction.
source: https://www.nisenet.org/catalog/scientific-image-quantum-corral-top-view

For the second bachelor students in chemistry the introduction to quantum chemistry finally put them into contact with some “real” quantum mechanics when they were introduced into the world of potential barriers, steps, and wells. Though these are still abstract and toy-model in nature, they provide a first connection with reality, where they can be seen as crude approximations of the potential experienced by valence electrons near the surface, or STM experiments. They were also introduced to my favorite quantum system related to this course: the quantum corral. Without any effort it can be used in half a dozen situations with varying complexity to show and learn basic quantum mechanics. For the third bachelor chemistry students the course quantum and computational chemistry finally provided them the long promised first example of a non-trivial quantum chemical object: The Helium atom. With it’s two electrons, we break free of the H-atom(-like)  world. Using perturbation theory and Slater determinant wave functions, we made our first approximations of its energy. In addition, these students also had a seminar for their course Introductory lectures in preparation to the bachelor project (Kennismakingstraject m.b.t. stage en eindproject, in Dutch). During this lecture I gave a brief introduction and overview of the work I did in the past and the work we do in our research group QuATOMs, which although “quantum” is quite different of what the students experience during their courses on quantum chemistry.

In the materiomics program, the first master students continued their travels into the basics of force-fields during the lecture of the course Fundamentals of materials modelling. The exercise class of this week upped the ante by moving from ASE to LAMMPS for practical modeling of alkane chains, which was also the topic of their second lab session. In the course Properties of functional materials, we investigated the ab initio modelling of vibrations. During the exercise classes we investigated precalculated phonon spectra in the materials-project database, as well as calculated our own vibrational spectrum at the gamma-point of the first Brillouin zone. During the second master course Machine learning and artificial intelligence in modern materials science the central theme was GIGO (Garbage-In-Garbage-Out). How can we make sure our data is suitable and good enough for our models to return useful results. We therefore looked into data-preparation & cleaning, as well as  clustering methods.

At the end of this week, we have added another 22h of live lectures and ~1h of video lectures, putting our semester total at 74h of live lectures. Upwards and onward to week 6.

Jul 18 2022

Cover image Polymer International: A machine learning approach for the design of hyperbranched polymeric dispersing agents based on aliphatic polyesters for radiation curable inks

Authors: Danny E.P. Vanpoucke, Marie A.F. Delgove, Jules Stouten, Jurrie Noordijk, Nils De Vos, Kamiel Matthysen, Geert G.P. Deroover, Siamak Mehrkanoon, and Katrien V. Bernaerts
Journal: Polymer International 71(8), i-i (2022)
doi: 10.1002/pi.6434
IF(2015): 3.213
export: bibtex
pdf: <PolymerInt>

Abstract

The cover image is based on the Research Article A machine learning approach for the design of hyperbranched polymeric dispersing agents based on aliphatic polyesters for radiation-curable inks by Danny E.P. Vanpoucke et al., https://doi.org/10.1002/pi.6378.  (The related paper can be found here.)

 

Cover Polymer International: Machine learning on small data sets, application on UV curable inks.

Cover Polymer International: Machine learning on small data sets, application on UV curable inks.

 

Jul 16 2022

A machine learning approach for the design of hyperbranched polymeric dispersing agents based on aliphatic polyesters for radiation curable inks

Authors: Danny E.P. Vanpoucke, Marie A.F. Delgove, Jules Stouten, Jurrie Noordijk, Nils De Vos, Kamiel Matthysen, Geert G.P. Deroover, Siamak Mehrkanoon, and Katrien V. Bernaerts
Journal: Polymer International 71(8), 966-975 (2022)
doi: 10.1002/pi.6378
IF(2021): 3.213
export: bibtex
pdf: <PI> (Open Access) (Cover Paper)

 

An ensemble based machine learning model for small datasets was used to predict the relationship between the dispersant structure and the pigment dispersion quality (particle size) for radiation curable formulations.
Graphical Abstract:An ensemble based machine learning model for small datasets was used to predict the relationship between the dispersant structure and the pigment dispersion quality (particle size) for radiation curable formulations.

Abstract

Polymeric dispersing agents were prepared from aliphatic polyesters consisting of δ-undecalactone (UDL) and β,δ-trimethyl-ε-caprolactones (TMCL) as biobased monomers, which are polymerized in bulk via organocatalysts. Graft copolymers were obtained by coupling of the polyesters to poly(ethylene imine) (PEI) in the bulk without using solvents. Different parameters that influence the performance of the dispersing agents in pigment based UV-curable matrices were investigated: chemistry of the polyester (UDL or TMCL), weight ratio of polyester/PEI, molecular weight of the polyesters and of PEI. The performance of the dispersing agents was modelled using machine learning in order to increase the efficiency of the dispersant design. The resulting models were presented as analytical models for the individual polyesters and the synthesis conditions for optimal performing dispersing agents were indicated as a preference for high molecular weight polyesters and a polyester dependent maximum weight ratio polyester/PEI.

Animation of TMCL model 6

Animation of TMCL model 6

 

Feb 16 2022

Wordle-mania: an opportunity for python programming and artificial intelligence.

Over that last few months the wordle game has become increasingly popular, with people sharing their daily feats on Twitter. Currently the game is hosted by the NY times which bought it the end of January 2022 from its creator.  The game is rather straightforward: you have 6 guesses to find a 5-letter English word. Every guess, the game tells you if a letter is (1) not in the word [grey], (2) in the word at a different position [yellow/orange], or (3) in the word at the exact same position [green].

Wordle 242 4/6
⬛⬛🟨⬛⬛
⬛🟨⬛⬛⬛
⬛⬛🟩🟨🟩
🟩🟩🟩🟩🟩		 An example of the result (as it looks when shared on Twitter). My first guess was the word “PIANO”, which means the A is in the word but at a different position. My second word, “QUERY”, adds the U to the list of letters that are present. With my third guess, “STUCK”, the position of the U and the K are fixed and we now also know the letter C is involved. At this point, I was also stuck, so I got some help of my wordle-assistant program, which taught me there could only be 1 word matching all the information we had: “CAULK“.

This seamlessly brings me to the central topic of this post: writing a program to help win this game as efficiently as possible. Not terribly original, but it’s a means to an end, as this simple project allows us to explore some more advanced topics in programming in python as well as artificial intelligence.

During this exploration I’ll  be including and updating a set of tutorials as well as this post. The python side of the project will focus on efficiency and easy of use and distribution, while the AI side will focus on smart ways predicting the best possible next guess. For the latter, an important caveat is that this means that the program should also work if you’re the last player living on earth, or if you decide to play wordle in a different language or a different number of letters. This means that creating a distribution of the  tweeted results of other players and comparing this with the complete set of brute-forced distributions to guess the wordle of the day in a single guess, would not satisfy my definition of AI. It is an interesting Big-data kaggle competition though.

Python Tutorials

  • Classes in Python. This tutorial provides a simple introduction in the concept of classes in python.
  • Child Classes. Continuing on the previous we deal now with child classes and the intricacies of function overriding and accessibility.
  • Python Library on Github.
  • Jupyter Notebook examples.
  • Slow Python: Profiling python in Jupyter. We look into profiling a Jupyter notebook script, to find the bottlenecks.
  • Slow Python: speeding up copying.

 

Artificial Intelligence

  • Distributions of words and letters.
  • Information theory of wordle?

 

The WordleAssistant Library and notebooks.

All tutorial code and jupyter notebooks can be found in this github repository.

May 06 2021

TEDx Talk: The Virtual Lab

Happy to announce my TEDxUHasselt talk is officially part of the TEDx universe:  https://www.ted.com/talks/danny_vanpoucke_the_virtual_lab .

I enjoyed talking about the VirtualLab. Showed examples from atoms to galaxies, from computer-chips to drug-design and from to opinion-dynamics to epidemiology. I looked at the past and and glanced towards the future, where machine learning and artificial intelligence are the new kids on the block.

 

Aug 04 2020

Practical Machine-Learning for the Materials Scientist

Scilight graphic

Individual model realizations may not perform that well, but the average model realization always performs very well.

Machine-Learning  is up and trending. You can’t open a paper, magazine or website without someone trying to convince you their new AI-improved app/service will radically change your life. It will make the production of your company more efficient and cheaper, make costumers flock to your shop and possibly cure cancer on the side. Also in science, a lot of impressive claims are being made. General promises entail that it makes the research of interest faster, better, more efficient,… There is, however, a bit of fine print which is never explicitly mentioned: you need a LOT of data. This data is used to teach your Machine-Learning algorithm whatever it is intended to learn.

In some cases, you can get lucky, and this data is already available while in other, you still need to create it yourself. In case of computational materials science this often means performing millions upon millions of calculations to create a data set on which to train the Machine-Learning algorithm.[1] The resulting Machine-Learning model may be a thousand times faster in direct comparison, but only if you ignore the compute-time deficit you start from.

In materials science, this is not only a problem for those performing first principles modeling, but also for experimental researchers. When designing a new material, you generally do not have the resources to generate thousands or millions of samples while varying the parameters involved. Quite often you are happy if you can create even a few dozen samples. So, can this research still benefit from Machine-Learning if only very small data sets are available?

In my recent work on materials design using Machine-Learning combined with small data sets, I discuss the limitations of small data sets in the context of Machine-Learning and present a natural approach for obtaining the best possible model.[2] [3]

The Good, the Bad and the Average.

(a) Simplified representation of modeling small data sets. (b) Data set size dependence of the distribution of model coefficients. (c) Evolution of model-coefficients with data set size. (d) correlation between model coefficient value and model quality.

In Machine-Learning a data set is generally split in two parts. One part to train the model, and a second part to test the quality of the model. One of the underlying assumptions to this approach is that each subset of the data set provides an accurate representation of the “true” data/model. As a result, taking a different subset to train your data should give rise to “the same model” (ignoring small numerical fluctuations). Although this is generally true for large (and huge) data sets, for  small data sets this is seldomly the case (cf. figure (a) on the side). There, the individual data points considered will have a significant impact on the final model, and different subsets give rise to very different models. Luckily the coefficients of these models still present a peaked distribution. (cf. figure (b)).

On the down side, however, if one isn’t careful in preprocessing the data set correctly, these distributions will not converge upon increasing the data set size, giving rise to erratic model behaviour.[2]

Not only the model coefficients give rise to a distribution, the same is true for the model quality. Using the same data set, but making a different split between training and test data can give rise to large differences in  quality for the model instances. Interestingly, the model quality presents a strong correlation with the model coefficients, with the best quality model instances being closer to the “true” model instance. This gives rise to a simple approach: just take many train-test splittings, and select the best model. There are quite some problems with such an approach, which are discussed in the manuscript [2]. The most important one being the fact that the quality measure on a very small data set is very volatile itself. Another is the question of how many such splittings should be considered? Should it be an exhaustive search, or are any 10 random splits good enough (obviously not)? These problems are alleviated by the nice observation that “the average” model shows not the average quality or the average model coefficients, but instead it presents the quality of the best model (as well as the best model coefficients). (cf. figure (c) and (d))

This behaviour is caused by the fact that the best model instances have model coefficients which are also the average of the coefficient distributions. This observation hold for simple and complex model classes making it widely applicable. Furthermore, for model classes for which it is possible to define a single average model instance, it gives access to a very efficient predictive model as it only requires to store model coefficients for a single instance, and predictions only require a single evaluation. For models where this is not the case one can still make use of an ensemble average to benefit from the superior model quality, but at a higher computational cost. 

References and footnotes

[1] For example, take “ANI-1: an extensible neural network potential with DFT accuracy at force field computational cost“, one of the most downloaded papers of the journal of Chemical Science. The data set the authors generated to train their neural network required them to optimize 58.000 molecules using DFT calculations. Furthermore, for these molecules a total of about 17.200.000 single-point energies were calculated (again at the DFT level). I leave it to the reader to estimate the amount of calculation time this requires.

[2] “Small Data Materials Design with Machine Learning: When the Average Model Knows Best“, Danny E. P. Vanpoucke, Onno S. J. van Knippenberg, Ko Hermans, Katrien V. Bernaerts, and Siamak Mehrkanoon, J. Appl. Phys. 128, 054901  (2020)

[3] “When the average model knows best“, Savannah Mandel, AIP SciLight 7 August (2020)

Aug 04 2020

Small Data Materials Design with Machine Learning: When the Average Model Knows Best

Authors:  Danny E. P. Vanpoucke, Onno S. J. van Knippenberg, Ko Hermans, Katrien V. Bernaerts, and Siamak Mehrkanoon
Journal: Journal of Applied Physics 128, 054901 (2020)
doi: 10.1063/5.0012285
IF(2019): 2.286
export: bibtex
pdf: <JApplPhys>   (Open Access)
github: <Amadeus>

 

Vulcanoplot
Graphical Abstract: Correlation plot of the RMSE of the validation set and the intercept value for linear model instances trained on 1000 subsets of a 25 point data set. The distribution of the correlation data is indicated by the black curve.

Abstract

Machine Learning is quickly becoming an important tool in modern materials design. Where many of its successes are rooted in huge data sets, the most common applications in academic and industrial materials design deal with data sets of at best a few tens of data points. Harnessing the power of Machine Learning in this context is therefore of considerable importance. In this work, we investigate the intricacies introduced by these small data sets. We show that individual data points introduce a significant chance factor in both model training and quality measurement. This chance factor can be mitigated by the introduction of an ensemble-averaged model. This model presents the highest accuracy while at the same time it is robust with regard to changing data set size. Furthermore, as only a single model instance needs to be stored and evaluated, it provides a highly efficient model for prediction purposes, ideally suited for the practical materials scientist.