Programming improvements 3

A few things of the many things I learned last months in Python.

For and while loops have an else option

I used to write code like this:

helper = False
for something in some_list:
    if condition:
       helper = True
       break
if helper == False:
   do_something()

That can be done so much easier:

for something in some_list:
    if condition:
       break
else:    # break wasn't called in the loop
    do_something

A similar option exist for while loops.

Few other little things were:

if helper == False:          ->            if not helper:

def method(somevar=""):      ->            def method(somevar=None):
    if somevar != "":                          if somevar:

List comprehensions were also a revelation. Instead of using:

newlist = []
for entry in some_list:
    if condition:
        newlist.append(entry)

condition could be something like “entry > 0”. Using a list comprehension, that can be a one liner and is also faster in the execution:

newlist = [entry for entry in some_list if condition]

While I still feel, it would have been nice, if I’d have learned these little things before (which comes from the thought I would be liked more, and from the expectation of perfectionism by me and because of that I also think others expect perfectionism from me – which is just not achievable), it is nice to pick new things up now. While I thought in my 20s that in many areas of life I can stop learning at some point, in the last years I learned that learning will never end in any part of life, and that is good! Because if I stop being willing to learn, I will go backwards.

Low sun in winter

During frosty days, just before the winter solstice, I came along this place.

Frosty meadow in England. The parts in the shade from the hedge have three days of frost to them, the other parts just one day.

The sun was shining the days before, and cleared the frost where it reached.

I was surprised, how wide the shady area was. My instinct would have been, that the shadow would just have been a bit wider than the height of the hedge. But it is much wider. In the picture I measured the width of the frosty area and the height of the hedge at a few places and calculated the angle of the suns maximum elevation using

alpha = arctan(height/width)

Height [pixel]	Width [pixel]	Angle alpha [degree]
344	1120	17.1
656	2080	17.5
1012	3612	15.7

Measured values and calculated angle of the sun

In the table above one has to take into account that it’s impossible to measure with one pixel precision for a hedge on uneven ground. Another person might measure the heights 30 pixel higher, or lower, and widths 80 pixel wider or narrower (for the second row), which could vary the angle by about 1.5 degrees. That was the reason, why I did three measurements of different areas in the picture.

So I get about 16 to 18 degrees, which corresponds with the real value of 90 – 51 – 23.4 = 15.6 degrees. The values in the formula are:
90: mid day height of the sun over the equator
51: my geographical latitude
23.4: angle between equator and ecliptic.
At the winter solstice the latter value needs to be subtracted, at the summer solstice it has to be added, giving a height of 62 degrees for the sun during mid day

Climate crisis at home (update)

Five and a half weeks after me writing about the dry weather in the UK we finally got some substantial rain (about 25 mm over 8 hours). After the rain I took another picture of the same hole as before:

Deep cracks in the soil in the garden on 25 August 2022. The remaining green grass is less green, trees are getting rid of their leaves due to drought. The hole has slightly filled up with soil falling of the cliffs.

The old picture:

Deep cracks in the soil in the garden, less than 10 cm of the 30 cm ruler stick out. At least some grass is still green, and we hope for rain before all died back. — Deep cracks in the soil in the garden on 17 July 2022. Less than 10 cm of the 30 cm ruler stick out of the crack. At least some grass is still green, and we hope for rain before all dies back.

I thought that the cracks got bigger, however, the pictures don’t show a clear widening of the crack. It’s not completely clear due to the different angles of the pictures. But still a good example why we shouldn’t rely on feelings or opinions when making decisions, but on measurable facts.

After the rain I also wanted to see how deep the moisture penetrated the soil, 6 hours after the end of the rain I dug a tiny hole. I was surprised how quickly the spade stopped, due to hitting dry soil:

The 25 mm of rain penetrated about 50 mm of soil, below the soil is still bone dry.

It will take a lot more rainy days before the moisture will reach depths of over a meter and even longer to fill up the aquifers. With the measurement from our garden today, I think we would need this amount of rain every day for at least a month to counteract the drought. Forecast predicts no rain for the next 7 days, so I guess the top layer of the soil will be again completely dry by the time the next rain will be here.

Below I copied the text from my original article. And I actually feel I was quite good with my actions. I kept cycling most local journeys and reduced my meat and milk consumption and just bought the train tickets to go to Germany in a month (which is not the easiest travel, but luckily there is help: https://www.seat61.com/Germany.htm#london-to-leipzig-and-dresden-by-train)

In future these droughts are likely to occur more often, the last IPCC report is very clear about this. And there are things we can do. Like planting some trees in the garden (and maybe on fields?). From my experience in Germany and the UK, people don’t like trees in the garden (all this work with the leaves, …) but in our garden there is a clear difference in the number and size of the cracks in the shade of the tree and in the grass area. And huge difference in grass colour. I hope we can convince our landlord, to get the approval for a few more trees.

The reason for the climate crisis now, is however the behaviour of the generations from the 1960 up to mine. An economy was build on cheap oil and gas, without paying the true costs (and now people complain, when they have to pay prices closer to the real costs). All the work that needs to go in moving cities away from the coast, to repair infrastructure after it was hit by a heat wave (or much heavier/frequent storms/floods), all the lives lost in heat waves and other severe weather. We are paying now for the living standard of our grandparents (And poor countries for the wealth of rich countries, the colonialism still continues). And a further problem is the inflexibility for change in these generations (don’t take away my petrol car or the 200 km/h on a German motorway, I want to drive everywhere, why do I need to see wind farms or solar parks in my neighbourhood, one day a week without meat? – How dare you!). Unfortunately, these are the people that make politics and decisions. And they (and me) will be dead when we reach the 4°C average warming by the end of the century (that is the scenario our current goals announced by the politics head to), and future generations will have to deal with that. Parents usually say they love their children, but somehow their actions feel different.

So what can I do:

do I need to use a car, or can I spent a few minutes longer on the public transport (less stress) or use an (electric) bicycle (saves the gym visit)?
can I eat more plant based products? (lentil burger/bolognese, oat milk, bean based spread instead of salami)
do I need to take the plane, or can I spent a bit longer, but use the train? (challenging)

Programming improvements 2

A few things of the many things I learned last weeks in Python.

This is what I used to do to conditionally set settings:

class MyClass:
    def method(self, args**):
        <some code block>
        if some_arg == "condition":
            self.option = "this setting"
            <run some code>
            <call a method>
        else:
            self.option = "another setting"
            <run some other code>
            <call a different method>
        <some more code block>

myclass = MyClass()
myclass.method(args)

And it could be much more complicated.

Using an object oriented approach, can be so much nicer.

class MyBaseClass:
    def method(self, args**):
        <some code block>
        self._submethod(subargs**)
        <some more code block>
    @abstractmethod
    def _submethod():
        pass

The private submethod would then be defined in the child classes. First for “condition”

class MyClass_condiontion(MyBaseClass):
    def _submethod()::
        self.option = "this setting"
        <run some code>
        <call a method>     # can be in this or the parent class

Second for the else:

class MyClass_else_condiontion(MyBaseClass):
    def _submethod()::
        self.option = "another setting"
        <run some other code>
        <call a different method>     # can be in this or the parent class

To call the classes one could use the condition:

if some_arg == "condition":
    myclass = MyClass_condiontion()
else:
    myclass = MyClass_else_condiontion()
myclass.method(args)

With this approach the coding is so much neater. Each method knows what it has to do and there is no fuss about the settings.

Climate crisis at home

Before I moved to the UK, I heard of the rainy, foggy weather here. Typical English weather. But I have to say, that is not at all my experience. During the cycle commutes to and from work I got wet rarely and only at the a similar rate as in Germany. Although, these cycles are slightly biased, as I check the rain radar and have stayed half an hour longer to let rain pass, or left work a bit earlier to get home before the rain clouds arrived.

Now it has been a very long time since I last was out in the rain. And that’s not because I haven’t been out. While most of June was relatively cool with many days below 20°C, it didn’t rain much. So far July was very dry as well, at higher temperatures. The really hot days, however, are still ahead.

The dry weather is definitely visible now. On my recent cycles I saw so many brown meadows or village greens. And the cracks in the soil in our garden get bigger and bigger. I never have seen such deep cracks in soil with vegetation on it.

So what can I do:

do I need to use a car, or can I spent a few minutes longer on the public transport (less stress) or use an (electric) bicycle (saves the gym visit)?
can I eat more plant based products? (lentil burger/bolognese, oat milk, bean based spread instead of salami)
do I need to take the plane, or can I spent a bit longer, but use the train? (challenging)

Programming improvements

A few things of the many things I learned last week in Python.

My draft when a new optional dictionary was added to the method:

def method(self, arg1, options={}):

However, a better version was suggested to me:

def method(self, arg1, options=None):
    options = (options or {})

And the other big learning through code review was, when constructing a string from a dictionary. My complicated code:

for key1, value1 in options.items():
    if key1 == "searchString":
        for key, value in value1.items():
            result_string = f"someString.{key}.{value}"

The suggested version is so much neater:

for key, value in options.get("searchString", {}).items():
    result_string += f"someString.{key}.{value}"

How to code, an additional note

(by Ronny Errmann)

My last post could be read quite negatively, in the way that I didn’t make any progress. So I wanted to add some notes to counteract that feeling. When I looked into new methods actively, because I had a problem, I learned a lot. And I easily remember a few bits from the last years:

Replacing loops and conditions with numpy operations on the whole array or part of the array. This made execution so much faster. For example using

array[array > 0] = numpy.average(another_array, axis=1)

Making my program compatible to Windows. It’s just a few things to keep Python code compatible to different operating systems.

Adding multiprocessing to the program.

All of these helped me to learn new concepts and improve.

How to code, to my younger me

(by Ronny Errmann)

Every time I prepared for interviews or when I started a new role, I learned so many new concepts. Unfortunately, once working in a scientific role, I found it harder to keep this learning up. Making the time to learn about concepts and to talk about code was quite low on my priority list. While reviewing code is something, which I still think is difficult, learning new concepts is something that can easily done. For example, freeing up one hour once or twice a week can make a big difference. Recently I look into C++ methods and just by looking through a few pages of a tutorial (e.g. about classes), I feel I increased my knowledge by a large amount. So there is really no reason to use that as an excuse.

Discussion about code is a bit more difficult. Being the sole programmer in a team doesn’t make things easier, discussion of code means I take time from a different project. Furthermore, in a scientific environment, most people were also self-taught and, like me, only learned the necessary information and were goal driven. But of course, it is important to discuss code and the few experiences, that I have had in the last months, together with the explanation how companies practise these discussions, have changed my view a lot. It is worth the effort, and try to find a group to discuss things can be really beneficial. So similarly to my last post, I would answer to the question, if I would do things differently now with a clear yes.

A beginners code

(by Ronny Errmann)

As mentioned in my last post, I started coding without asking for advice. Which meant I would do things inefficiently. For example to calculate the standard deviation I used the following code:

mittel=sum(l for l in temp)/len(temp)
stdev=math.sqrt(sum((l-mittel)**2 for l in temp)/(len(temp)-1))

Later I learned about numpy, which made things a bit easier:

stdev = numpy.std(temp, ddof=1)

This is also quite a bit more efficient in computing time as it is run in compiled code and not has the overhead of an interpreter language.

Looking back I can smile about the code. In an interview I was once asked about that program if I would do things differently now. To which my answer was a clear yes.

How I came to writing code

(by Ronny Errmann)

The first device I wrote my first programs was the CASIO CFX-9850G, a graphical calculator. We needed it for school (I think 8th grade) and to get it for a lower price, were suggested to buy it even half a year earlier. That was in the mid 90s and, out of interest and out of boredom, I read the manual to see what one can do with it. And by talking to friends, being showed programs from others, I started to understand other people’s code and then tried to solve smaller questions. With time the programs got bigger. In the end I programmed a battleship game and 4 wins, ready to be played in boring lessons against the person sitting next to you. While the programs are long gone (maybe the hand-copied code is still in a folder at my parents place), the calculator is still a part of my desk, although, primarily just as a calculator.

A few years later (or at the same time?) we learned Delphi in School. It was only one year with one or two hours a week, so it couldn’t go into details, however, I know I tried to improve the programs and write more complex ones after school or at home. The GUI made some parts easy, however, looking back it also prevented me from understanding that there is a big benefit for writing back-end code.

After School my interest in coding decreased, mostly because I thought I wasn’t good enough or didn’t see the application. And interest in learning background theory was limited at that age. When I started my Physics studies live was also busy enough. To analyse the laboratory experiments I would work with Excel/OpenOffice Calc and Origin. The built-in functions were enough to solve everything, not elegant, but efficient (in terms of my time).

In my masters project I found the limitations of OpenOffice Calc. To do calculations I just needed so many columns that I would crash it on a regular basis. However, it took until I had to do the first analysis of observational data, that I noticed I need to learn programming. Other people at the institute suggested Python as a start. And so my first basic program was just a python script that would do the steps I would do by hand by itself. It was very procedural, and when I look back, quite funny in how I wrote it. I definitely didn’t search for advice. Luckily this has changed over the years.