OZM Trainspotting
A little "AI for Good" story
This week we have that OZM added a new page. The idea of collecting train graffiti came up a long time ago, but we didn't have the time to really look into it. We've come back to the subject on several occasions, but never really. Until August 2021 when we decided that this is the perfect job for an AI.
There is a ingrained injustice in art that is not uncommonly addressed in literature and music. Art requires not only talent and skill, but also time. Much time. To be recognized as an artist, you need the right canvas and exhibition so people have a chance to see and appreciate your art (or not). But such a canvas is hard to come by, let alone an exhibition. Exhibition places are rare and there is fierce competition among the competitors, which most of them do not judge impartially, to put it mildly.
Street artists realized that they were given an easily available canvas with an exhibition, albeit short-term, in the form of the public transit system. Although some would perhaps see "given" as an unfortunate, if at all legitimate, exaggeration.
The OZM has the privilege of having one of the busiest railway lines in front of its door. Thus it has the opportunity to offer an exhibition platform for fine street art in Germany. If you're on the roof of the OZMs sitting and watching the train lines, you will see amazing works of art. We don't know if those in charge of the major rail companies have a precise idea of what a train should look like, or if they just want to provide a clean canvas for the next artist. But these masterpieces rarely survive more than a few days, so most trains look the same afterwards. So looking at the train lines all day is pretty boring, even if you really like trains. But for an AI, this is an ideal job because it is impartial and doesn't mind watching the train lines all day long.
We had previously trained an artificial neural network (ANN) for street art recognition and reduced the network size to 1,3 MB, making it well suited for real-time recognition. So this ANN should be our unbiased judge, which of course poses a problem since the choice of training data contains bias induced by our own biases.
Anticipating this problem, the network was initially trained with a mix of web crawling graffiti and street art images collected largely independently of artistic creation height (or depth) in different neighborhoods of Hamburg.
The network trained on this database was fed several hours of videos of train graffiti and collected thousands of training examples of train graffiti as well as some funny so-called false positives. Since we were only interested in the "street art-like" images, we used a combination of network output activation and interlayer cosine spacing to filter the videos for the best shots of all masterpieces on display. These training examples have been added to the network's training database, giving a good starting point for unbiased training.
First we placed a camera with a 10x optical zoom connected to a small video processing computer (actually a Jetson Nano) on the second floor of the OZMs to observe the railway lines and send the video to our inference engine for analysis. So now we had an AI that gave us snapshots of the graffiti on the trains.
We only used the Nano to do some video encoding and send the video over WiFi to our little inference engine where we did the same thing we did with the videos. Of course we had some problems with real negatives that were not included in the training data, like video decompression artifacts due to poor wifi quality or some trains like the ICE that were not in the database. The perspective from the camera was also very different from the usual training examples. We solved this by freezing the feature detection layers and adjusting the top layers of the mesh with additional samples collected in the first days of testing.
But it's not just about taking the best snaps. It is also about the presentation of the snapshots, which should show our respect and appreciation for the works presented and the effort and commitment that made them possible.
We wanted to add something to the photos of the works to show the relationship between the graffiti on the trains and the art from the OZM being able to symbolize being made by artists who have started their careers working on the railway lines just outside OZM had started. Only graffiti from the OZM relying on the pictures would have been lame. Making new graffiti for every picture we find would not have been possible for organizational reasons. Again, AI was the solution.
To add something appropriate, we trained various AIs on works by established street artists who are familiar with the OZM collaborate and compare the results. We decided to combine the selected photos with the original Hammerbrooklyn-Sign a logo filled with images of so-called “latent walks” of a style goose2. Such a picture can be seen here on the far left. That's in the middle Hammerbrooklyn-Recognizable logo. On the snapshot, the outline of the Hammerbrooklyn-Logos then used as a template to create the generated image of the AI in the form of the Hammerbrooklyn- “Stamp” logos into the snapshot.
The next big question was which snaps should be presented and how. We agreed on random snaps sorted by day of the week. However, to encourage visitors to browse the images, we have hidden most of the images and used the weekday names as containers. When a visitor mouses over a letter, that part becomes visible, but to see the whole picture, visitors have to click on the letters.
In order to let visitors know the actual date the photo was taken, it was supposed to be printed on the snapshot, which was a bit problematic. Every snapshot shows individual artworks, because we understand that OZM as an art space. So simply rendering the date with a font wasn't an option. A graffiti of the date would be the best solution, but it had to be unique. A third time the solution was the AI.
We added the date in the form of a graffiti generated by an AI, which in turn was trained on shapes from OZ's works. The graffiti doesn't look like what most people would expect from OZ, who was famous for his Ozozozozoz writings. But copying OZ was never the aim, rather creating a new style heavily influenced by OZ's late work. Anyone familiar with OZ's late works will notice the similarity.
The method takes digits rendered in a common font (below right), warps them, and performs some boundary detection. The resulting image (large left image) is fed to a pix2pix gan that has been extensively trained on distorted shapes extracted from images from OZ, but with the detail removed from the input image.
We consider this an example of “AI for Good” for several reasons. Nobody lost their job. The AI collects and preserves art that would otherwise have been lost. The art collected by the AI is presented to a wider audience. The AI is impartial to artists and styles, so no one is disadvantaged. OZ's style is echoed in the images to commemorate the late Hamburg graffiti artist who never gave up beautifying his city and thus made an important contribution to the variety of works that we can admire on the trains today.
* The snapshots are on https://onezeromore.com/ozmai-2/trainspotting/ displayed. We encourage you to browse the images as this will allow you to enjoy all of the art. We're also launching an Instagram account for your daily dose of new train graffiti arrivals.
If you find a particularly appealing graffiti that you would like to have on your own train, building, truck, boat, plane, etc., we are happy to help you. Just contact us.
* Of course we do not know the author of the pictures. But usually we know artists who are able to create an interpretation of graffiti that is very close to the original. (Due to the resolution of the camera and the size of the graffiti on the recordings, a 1 to 1 copy would be pixelated anyway).
