Rainfall / Regen (Analysis / Auswertung)

Introduction to the topic and the process in the previous post.Einleitung in das Warum und Wie im letzten Beitrag.
In the next image it indeed looks like that it rains less at home (the white dot in the centre; the box is 21 x 21 pixel or 7 x 7 km) than in the area around it. For each month the central area received less rain than the surroundings.Im nächsten Bild sieht tatsächlich so aus, als ob es zu Hause (der weiße Punkt im Zentrum; die Box ist 21 x 21 pixel, 14 x 14 km) weniger regnet. Für jeden der abgedeckten Monate gibt es in der Bildmitte einen trockeneren Bereich.
Monthly Summary of the rainfall radar data at my home in a 14x14 km box.
That pattern is visible in the weekly summarised data for some weeks, but not in the 18 weeks of daily data below (except for 3 days mid October).Dieses Muster ist in den wochenweise zusammengefassten Daten in manchen sichtbar, jedoch nicht in 18 Wochen von täglichen Zusammenfassungen (mit Ausnahme von 3 Tagen Mitte Oktober).
Daily Summary of the rainfall radar data at my home in a 14 x 14 km box.
Daily Summary of the rainfall radar data at my home in a 14 x 14 km box.
So it seemed that indeed it rains less at home than a bit further away. But the similar pattern of this line from southwest to northeast made me doubt it. The prevailing wind is that direction, but there is no feature in geography that would explain it.Es wirkt tatsächlich so, als ob es Zuhause weniger regnet als ein paar Kilometer weiter östlich oder westlich. Aber das ähnliche Muster von Südwesten nach Nordosten machte mich unsicher. Die dominante Windrichtung ist zwar in diese Richtung, allerdings gibt es um den Start herum keine markante Änderung in Geografie, die das Muster erklären würde.
So I added another region: 100 × 100 km around home. The next image shows the monthly summaries. And there “rays” coming from the radar station in the lower-left corner become visible. The calibration between radar echo strength and rainfall seems to be incorrect in some directions and assume too little attenuation. One of these “rays” just happens to pass right over the garden.Also habe ich nochmal eine weitere Region dargestellt: 100 x 100 km ums Zuhause. Im nächsten Bild sind die monatlichen Zusammenfassungen. Und da sieht man “Strahlen” von der Radarstation in der unteren linken Ecke kommen. Die Kalibrierung zwischen Stärke des Radarechos und Regen scheint in ein paar Richtungen nicht korrekt zu sein und zu wenig Dämpfung anzunehmen. Einer dieser “Strahlen” geht zufällig über den Garten.
Monthly Summary of the rainfall radar data at my home in a 100 x 100 km box.
So it seems the garden gets as much water as the surrounding area. It will be interesting how late spring and early summer look next year, as I only started gathering data by mid July 2025.Der Garten scheint also genauso viel Regen wie die Umgebung abzubekommen. Mal sehen wie der späte Frühling und Sommer nächstes Jahr aussehen, da meine Datensammlung erst Mitte Juli 2025 anfängt.
On some days you can indeed see that the rain passed by: July 17, 20, 21, 27, and 29, as well as August 3 and September 11 and 13; however, there are also days that show the opposite.An einigen Tagen sieht man tatsächlich, dass der Regen vorbei gezogen ist: 17., 20., 21., 27. und 29. Juli, sowie 3.8., 11. und 13.9., allerdings gibt es auch entgegengesetzte Tage.

Rainfall / Regen (Introduction / Einführung)

English below

In den vergangenen Sommern wirkte es so, als ob der Regen des öfteren einen Bogen um den Garten machte. Mal regenete es ein paar Kilometer westlich, mal zog die Regenwolke südlich vorbei, oder auf dem Weg zur Arbeit waren die Straßen überall nass, nur nicht im Heimatort. Ich habe mich da gelegendlich geärgert, da Giesen Arbeit ist, und besonders nervig war es, wenn die Regentonnen leer wurden.

Und dann wollte ich im Sommer in Schottland wandern gehen und sehen, wie feucht der Sommer bisher war. Der britische Wetterdienst Metoffice hält einige Daten vor, allerdings nicht so detailiert wie ich mir wünschte (Gerasterte monatliche Zusammenfassungen für die vergangenen Jahre: https://data.ceda.ac.uk/badc/ukmo-hadobs/data/insitu/MOHC/HadOBS/HadUK-Grid/v1.3.1.ceda/5km/rainfall/mon-30y/ , Daten von Wetterstationen: https://catalogue.ceda.ac.uk/uuid/dbd451271eb04662beade68da43546e1/ oder Extreme: https://weather.metoffice.gov.uk/observations/weather-extremes). Ich wollte jedoch sehen, ob es stimmt, dass die Regenwolken einen Bogen um den Garten machen, und dafür waren die frei zugänglichen Daten einfach nicht detailiert genug. Die Lösung für mich waren die Regenradardaten: https://weather.metoffice.gov.uk/maps-and-charts/rainfall-radar-forecast-map#?map=Rainfall. Dort gibt es die Beobachtungsdaten der letzten zwei Tage. Die Browser Developmenttools zeigten auch direkt, dass die Regendaten nur ein Bild (aller 5 minuten) sind:

https://maps.consumer-digital.api.metoffice.gov.uk/wms_ob/single/high-res/rainfall_radar/YYYY-MM-DDTHH:MM:00Z.png

Also kleines Programm geschrieben, was die Daten runterläd und speichert, und dann über die nächsten Monate mehr und mehr Auswertung hinzugefügt. Das Projekt ist auf Github verfügbar: https://github.com/ronnyerrmann/rainfall.

Die Auswertung ist im nächsten Artikel.

In den vergangenen Sommern wirkte es so, als ob der Regen des öfteren einen Bogen um den Garten machte. Mal regenete es ein paar Kilometer westlich, mal zog die Regenwolke südlich vorbei, oder auf dem Weg zur Arbeit waren die Straßen überall nass, nur nicht im Heimatort. Ich habe mich da gelegendlich geärgert, da Giesen Arbeit ist, und besonders nervig war es, wenn die Regentonnen leer wurden.

Und dann wollte ich im Sommer in Schottland wandern gehen und sehen, wie feucht der Sommer bisher war. Der britische Wetterdienst Metoffice hält einige Daten vor, allerdings nicht so detailiert wie ich mir wünschte (Gerasterte monatliche Zusammenfassungen für die vergangenen Jahre: https://data.ceda.ac.uk/badc/ukmo-hadobs/data/insitu/MOHC/HadOBS/HadUK-Grid/v1.3.1.ceda/5km/rainfall/mon-30y/; Daten von Wetterstationen: https://catalogue.ceda.ac.uk/uuid/dbd451271eb04662beade68da43546e1/ oder Extreme: https://weather.metoffice.gov.uk/observations/weather-extremes). Ich wollte jedoch sehen, ob es stimmt, dass die Regenwolken einen Bogen um den Garten machen, und dafür waren die frei zugänglichen Daten einfach nicht detailiert genug. Die Lösung für mich waren die Regenradardaten: https://weather.metoffice.gov.uk/maps-and-charts/rainfall-radar-forecast-map#?map=Rainfall. Dort gibt es die Beobachtungsdaten der letzten zwei Tage in 5-Minuten Auflösung mit einer Datenblockgröße von weniger als einem Kilometer. Die Browser Developmenttools zeigten auch direkt, dass die Regendaten nur ein Bild (aller 5 minuten) sind:

https://maps.consumer-digital.api.metoffice.gov.uk/wms_ob/single/high-res/rainfall_radar/YYYY-MM-DDTHH:MM:00Z.png

Also kleines Programm geschrieben, was die Daten runterläd und speichert, und dann über die nächsten Monate mehr und mehr Auswertung hinzugefügt. So werden die Daten für jeden Tag, Woche, Monat und Jahr aufaddiert. Für verschiede Gegenden werden die Daten extrahiert und als pdf, sowie Zeitraffer dargestellt. Das Projekt ist auf Github verfügbar: https://github.com/ronnyerrmann/rainfall.

Die Auswertung ist im nächsten Artikel.


In recent summers it often seemed as if the rain was making a detour around the garden. Sometimes it rained a few kilometers to the west, sometimes the rain cloud passed by to the south, or on the way to work the roads were wet everywhere—just not in my home town. When it was like this, it felt annoying, because watering is work, and it was especially frustrating when the rain barrels ran dry.

Then, in summer, I wanted to go hiking in Scotland and see how wet the summer had been so far. The British weather service, the Met Office, provides some data, but not in as much detail as I would have liked (gridded monthly summaries for past years: https://data.ceda.ac.uk/badc/ukmo-hadobs/data/insitu/MOHC/HadOBS/HadUK-Grid/v1.3.1.ceda/5km/rainfall/mon-30y/; weather-station data: https://catalogue.ceda.ac.uk/uuid/dbd451271eb04662beade68da43546e1/; or extremes: https://weather.metoffice.gov.uk/observations/weather-extremes). However, I wanted to find out whether it was really true that rain clouds were skirting around the garden, and for that the freely available data were simply not detailed enough.

The solution for me was the rainfall radar data: https://weather.metoffice.gov.uk/maps-and-charts/rainfall-radar-forecast-map#?map=Rainfall. There you can find observation data for the last two days, in 5 minute time resolution with one pixel showing less than a kilometre. The browser developer tools also showed immediately that the rain data are just images (one every 5 minutes):

https://maps.consumer-digital.api.metoffice.gov.uk/wms_ob/single/high-res/rainfall_radar/YYYY-MM-DDTHH:MM:00Z.png

So I wrote a small program that downloads and stores the data, and over the following months I added more and more analysis. The data are accumulated for each day, week, month, and year. For different regions, the data are extracted and presented both as PDFs and as time-lapse video. The project is available on GitHub: https://github.com/ronnyerrmann/rainfall.

The analysis is presented in the next article.






Programming improvements 3

A few things of the many things I learned last months in Python.

For and while loops have an else option

I used to write code like this:

helper = False
for something in some_list:
    if condition:
       helper = True
       break
if helper == False:
   do_something()

That can be done so much easier:

for something in some_list:
    if condition:
       break
else:    # break wasn't called in the loop
    do_something

A similar option exist for while loops.

Few other little things were:

if helper == False:          ->            if not helper:

def method(somevar=""):      ->            def method(somevar=None):
    if somevar != "":                          if somevar:

List comprehensions were also a revelation. Instead of using:

newlist = []
for entry in some_list:
    if condition:
        newlist.append(entry)

condition could be something like “entry > 0”. Using a list comprehension, that can be a one liner and is also faster in the execution:

newlist = [entry for entry in some_list if condition]

While I still feel, it would have been nice, if I’d have learned these little things before (which comes from the thought I would be liked more, and from the expectation of perfectionism by me and because of that I also think others expect perfectionism from me – which is just not achievable), it is nice to pick new things up now. While I thought in my 20s that in many areas of life I can stop learning at some point, in the last years I learned that learning will never end in any part of life, and that is good! Because if I stop being willing to learn, I will go backwards.

Programming improvements 2

A few things of the many things I learned last weeks in Python.

This is what I used to do to conditionally set settings:

class MyClass:
    def method(self, args**):
        <some code block>
        if some_arg == "condition":
            self.option = "this setting"
            <run some code>
            <call a method>
        else:
            self.option = "another setting"
            <run some other code>
            <call a different method>
        <some more code block>

myclass = MyClass()
myclass.method(args)

And it could be much more complicated.

Using an object oriented approach, can be so much nicer.

class MyBaseClass:
    def method(self, args**):
        <some code block>
        self._submethod(subargs**)
        <some more code block>
    @abstractmethod
    def _submethod():
        pass

The private submethod would then be defined in the child classes. First for “condition”

class MyClass_condiontion(MyBaseClass):
    def _submethod()::
        self.option = "this setting"
        <run some code>
        <call a method>     # can be in this or the parent class

Second for the else:

class MyClass_else_condiontion(MyBaseClass):
    def _submethod()::
        self.option = "another setting"
        <run some other code>
        <call a different method>     # can be in this or the parent class

To call the classes one could use the condition:

if some_arg == "condition":
    myclass = MyClass_condiontion()
else:
    myclass = MyClass_else_condiontion()
myclass.method(args)

With this approach the coding is so much neater. Each method knows what it has to do and there is no fuss about the settings.

While I still feel, it would have been nice, if I’d have learned these little things before (which comes from the thought I would be liked more, and from the expectation of perfectionism by me and because of that I also think others expect perfectionism from me – which is just not achievable), it is nice to pick new things up now. While I thought in my 20s that in many areas of life I can stop learning at some point, in the last years I learned that learning will never end in any part of life, and that is good! Because if I stop being willing to learn, I will go backwards.

Programming improvements

A few things of the many things I learned last week in Python.

My draft when a new optional dictionary was added to the method:

def method(self, arg1, options={}):

However, a better version was suggested to me:

def method(self, arg1, options=None):
options = (options or {})

And the other big learning through code review was, when constructing a string from a dictionary. My complicated code:

for key1, value1 in options.items():
    if key1 == "searchString":
        for key, value in value1.items():
            result_string = f"someString.{key}.{value}"

The suggested version is so much neater:

for key, value in options.get("searchString", {}).items():
    result_string += f"someString.{key}.{value}"

While I still feel, it would have been nice, if I’d have learned these little things before (which comes from the thought I would be liked more, and from the expectation of perfectionism by me and because of that I also think others expect perfectionism from me – which is just not achievable), it is nice to pick new things up now. While I thought in my 20s that in many areas of life I can stop learning at some point, in the last years I learned that learning will never end in any part of life, and that is good! Because if I stop being willing to learn, I will go backwards.

How to code, an additional note

(by Ronny Errmann)

My last post could be read quite negatively, in the way that I didn’t make any progress. So I wanted to add some notes to counteract that feeling. When I looked into new methods actively, because I had a problem, I learned a lot. And I easily remember a few bits from the last years:

  • Replacing loops and conditions with numpy operations on the whole array or part of the array. This made execution so much faster. For example using
array[array > 0] = numpy.average(another_array, axis=1)
  • Making my program compatible to Windows. It’s just a few things to keep Python code compatible to different operating systems.
  • Adding multiprocessing to the program.

All of these helped me to learn new concepts and improve.

A beginners code

(by Ronny Errmann)

As mentioned in my last post, I started coding without asking for advice. Which meant I would do things inefficiently. For example to calculate the standard deviation I used the following code:

mittel=sum(l for l in temp)/len(temp)
stdev=math.sqrt(sum((l-mittel)**2 for l in temp)/(len(temp)-1))

Later I learned about numpy, which made things a bit easier:

stdev = numpy.std(temp, ddof=1)

This is also quite a bit more efficient in computing time as it is run in compiled code and not has the overhead of an interpreter language.

Looking back I can smile about the code. In an interview I was once asked about that program if I would do things differently now. To which my answer was a clear yes.

How I came to writing code

(by Ronny Errmann)

The first device I wrote my first programs was the CASIO CFX-9850G, a graphical calculator. We needed it for school (I think 8th grade) and to get it for a lower price, were suggested to buy it even half a year earlier. That was in the mid 90s and, out of interest and out of boredom, I read the manual to see what one can do with it. And by talking to friends, being showed programs from others, I started to understand other people’s code and then tried to solve smaller questions. With time the programs got bigger. In the end I programmed a battleship game and 4 wins, ready to be played in boring lessons against the person sitting next to you. While the programs are long gone (maybe the hand-copied code is still in a folder at my parents place), the calculator is still a part of my desk, although, primarily just as a calculator.

A few years later (or at the same time?) we learned Delphi in School. It was only one year with one or two hours a week, so it couldn’t go into details, however, I know I tried to improve the programs and write more complex ones after school or at home. The GUI made some parts easy, however, looking back it also prevented me from understanding that there is a big benefit for writing back-end code.

After School my interest in coding decreased, mostly because I thought I wasn’t good enough or didn’t see the application. And interest in learning background theory was limited at that age. When I started my Physics studies live was also busy enough. To analyse the laboratory experiments I would work with Excel/OpenOffice Calc and Origin. The built-in functions were enough to solve everything, not elegant, but efficient (in terms of my time).

In my masters project I found the limitations of OpenOffice Calc. To do calculations I just needed so many columns that I would crash it on a regular basis. However, it took until I had to do the first analysis of observational data, that I noticed I need to learn programming. Other people at the institute suggested Python as a start. And so my first basic program was just a python script that would do the steps I would do by hand by itself. It was very procedural, and when I look back, quite funny in how I wrote it. I definitely didn’t search for advice. Luckily this has changed over the years.