Wölfischhttps://woelfisch.de/blog/2016-01-06T15:31:23+00:00Jörg's BlogMore fun with OpenLayers 3: U.S. National Parks2016-01-06T15:31:23+00:00o'wolfhttps://woelfisch.de/blog/author/owolfhttps://woelfisch.de/blog/more-fun-with-openlayers-3-us-national-parks<p>I've made a new interactive map while I was playing with the GIS stuff again. This time to visualize the U.S. National Parks, National Monuments, National Recreational Areas, and National Seashores / Lake Shores I've visited over the last eleven years.</p>
<p>Compared to previous versions, <a href="http://openlayers.org/">Openlayers 3</a> is refreshingly consistent, even though the asynchronous handling of data sources makes it somewhat cumbersome at times. For example, the geographic information at <a href="http://www.naturalearthdata.com/">Natural Earth</a> is split into three databases: areas for large parks, lines for sea and lake shores and points for smaller parks. Also, it contains too much data, but I haven't found a way how to filter it just for the parks I want to show and unify everything in one layer. Thus, the style callback has to do that. A rather ugly hack.</p>
<p>Also, it is not easy to find out how to convert the shape files from Natural Earth and the <a href="https://www.census.gov/geo/maps-data/">U.S. Census Bureau</a> from dBase on-disk format (is <em>anyone</em> here old enough to remember dBase?) to something OpenLayers can process. Fortunately, there is a free converter: <a href="http://www.gdal.org/">GDAL</a> comes with <tt>ogr2ogr</tt> that can even filter based on SQL queries. Recent versions support writing GeoJSON: <tt>ogr2ogr -f GeoJSON states.json cb_2014_us_state_20m.shp</tt>. Conversion to GPX is incomplete, it writes KML correctly but <a href="http://sourceforge.net/projects/viking/">Viking</a> isn't able to render it correctly, and the OpenLayers 3 KML format processor cannot cope with folders. Thus, I'm using GeoJSON.</p>
<p>Anyway, you can zoom and pan the map, and hovering with the mouse pointer over one of the yellow circles will pop up a small info box with the name of the park and the year(s) I visited it. It also works on mobile devices, however it may be a bit hard to successfully hit the circles on a touch screen.</p>
<p><iframe height="480" src="https://yaina.de/static/gps/us-parks.html" width="740">
<a href="https://yaina.de/static/gps/us-parks.html"><img height="460" src="https://yaina.de/static/gps/us-parks.jpg" width="720"/></a></iframe><br/><em>U.S. National Parks, Monuments, Recreational Areas, Seashores, Lakeshores visited</em></p>More fun with OpenLayers2008-11-26T20:37:00+00:00o'wolfhttps://woelfisch.de/blog/author/owolfhttps://woelfisch.de/blog/more-fun-with-openlayers<p>While browsing through the documentation of <a href="http://www.tangogps.org/">tangoGPS</a> I found the the relief map at <a href="http://www.maps-for-free.com/">maps-for-free.com</a>. The map is free (licenced under the <a href="http://www.gnu.org/copyleft/fdl.html">GNU FDL</a>) and generated from the <a href="http://www2.jpl.nasa.gov/srtm/">Shuttle Radar Topography Mission (SRTM)</a> <a href="ftp://e0srp01u.ecs.nasa.gov/srtm">data</a>:</p><p><a href="http://tracks.yaina.de/usa-2008.html?zoom=5&lat=36.84236&lon=-115.65969&layers=000BTT&poi=2008-01-30T22:23:41Z"><img height="160" src="http://img.woelfisch.de/maps4free-layer.jpg" width="238"/><br/><em>2008 RV trip, relief map</em></a></p><p>The <a href="http://www.openlayers.org">OpenLayers</a> integration was a bit more tricky than expected, as Hans' tile numbering is a bit different from the OSM scheme. But thanks to the example at <a href="http://petersmaps.googlepages.com/">Peter's Maps</a> I was able to integrate it.</p>Rendering GPS tracks2008-11-26T14:29:00+00:00o'wolfhttps://woelfisch.de/blog/author/owolfhttps://woelfisch.de/blog/rendering-gps-tracks<p>Sure, you can chose from numerous implementations if you want to render GPS tracks. But for scalable online maps, nothing I found was good for my purposes. The available on-demand renderers are too slow and need too much memory processing a data set with aproximately 100,000 fixes from all over California, Nevada, Arizona and Utah. Thus, I had to render the tiles offline. Basically, two free solutions for that exist: Mapnik and Osmarender.</p><p>Mapnik is terribly overblown for this purpose. It needs Postgresql, the PostGIS addon for psql, Boost, Python bindings to Boost, and lots of other libraries and tools I never used before. Worse, it is a really bad <a href="http://kylflow.livejournal.com/60126.html">dependency nightmare</a>.</p><p>Osmarender sounds more promising: basically, it is just an XSLT stylesheet that produces an SVG file from an OpenStreetMap file. To generate tiles, you need a filter to limit the OSM data to the region you want to render, an SVG renderer and something to cut the resulting large image into tiles. The Tiles@Home project provides the necessary scripts. However, it is specifically written to render and upload OSM tiles and it would need some tweaks and additional scripts to render custom data. I gave it a try again a couple of days ago, and all it rendered were broken PNG graphics. Apparently some issue with Inkscape, but I didn't bother to follow up on that.</p><p>With both Mapnik and Osmarender not being suitable for this specific job, I decided to brush up my Java knowledge and wrote my own renderer. Java Advanced Imaging (JAI) sounds like a good idea, as it can operate on tiled images. It is even possible to create your own tile handling. However, apart from the API and some far too simple tutorials, I didn't find any useful documentation on how to actually do that. In the end wrote my own tiled canvas implementation after wasting two days by searching for any kind of documentation. It allocates tiles only when they are actually used to save memory and can draw pixmaps and lines even over tile boundaries. The tiles are organized in a HashMap, which is fast and has an acceptable memory overhead. Rendering the track at zoom level 18 requires approximately 3.7 GB of RAM, which is 78 kB per tile, or 22% overhead. Could be better, but I expected worse.</p><p>Rendering 49,895 tiles took almost half an hour, which indeed is slower than anticipated. Profiling shows that most of the time gets spent internally in the Java AWT, mainly while converting the RGBA byte field to the internal raster and during PNG encoding. The latter is ridiculously slow. Sure, opposed to JPEG and TIFF processing which is implemented by C bindings, the PNG support was implemented purely in Java. Still it is no excuse to not fix it for more than ten years. What a shame, nobody tries to improve AWT anymore. The slowness of it is the main reason for all the prejudice against Java, after all.</p><p>Another time consuming task is the XML processing of the GPX file. I'm not satisfied with the speed of the SAX2 parser, but in this case it is at least acceptable. On the other hand, I'm positively surprised by the performance of the HashMap. Profiling showes that it even caches the last fetched item. I kept my own check though, as creating the string that serves as a hash key is notably more expensive than comparing two integer values...</p><p>If you want to know more about the program, or even like to extend it or fix bugs: It is GPLed and <a href="http://tracks.yaina.de/source/trackrenderer.tar.bz2">downloadable</a> from <a href="http://tracks.yaina.de/">tracks.yaina.de</a>. It is a command line tool, the source code is the documentation. Beware of the abominable coding style, though.</p>I'm addicted2008-10-22T20:56:00+00:00o'wolfhttps://woelfisch.de/blog/author/owolfhttps://woelfisch.de/blog/im-addicted<p>I'm addicted to geo-location and geo-tagging now. I cannot stop playing with it. I've geo-tagged my vacation photos from January now and built a small JavaScript application based on the incredible <a href="http://www.openlayers.org/">OpenLayers</a> framework to show where I took photos on an <a href="http://www.openstreetmap.org">OpenStreetMap</a> map or on top of <a href="http://onearth.jpl.nasa.gov/">Landsat7 satellite images</a>:</p><p><a href="http://tracks.yaina.de/usa-2008.html" target="_blank"><img height="421" src="http://yaina.de/~jreuter/lj/usa-2008-photos.jpg" width="381"/><br/>RV trip 2008 with my parents</a></p><p>And while I was at it, the same for the RV trip 2006 with Kayjay and <a href="http://users.livejournal.com/lynard_/">lynard_</a>:</p><p><div style="width: 490px;"><div style="float: left"><a href="http://tracks.yaina.de/usa-2006-tracks.html" target="_blank"><img height="161" src="http://yaina.de/~jreuter/lj/usa-2006-track.jpg" width="241"/><br/>Just the track</a></div><div style="float: right"><a href="http://tracks.yaina.de/usa-2006.html" target="_blank"><img height="160" src="http://yaina.de/~jreuter/lj/usa-2006-photos.jpg" width="241"/><br/>With photos</a></div></div></p><p><div style="clear: both"><br/>Works at least with Firefox 3, Konqueror 3, Opera 9, Safari 3.1, IE 7. Does not work with IE 6 and some builds of Konqueror 4. Google Chrome has issues.</div></p>Geotagging Issues2008-10-18T16:24:00+00:00o'wolfhttps://woelfisch.de/blog/author/owolfhttps://woelfisch.de/blog/geotagging-issues<p>The bad: discovering after months that the clock of the camera was set to CEST instead of UTC<br/>The good: exiftool and some few lines of perl to the rescue!</p><p><pre>#!/usr/bin/perl<br/>use strict;<br/>use Image::ExifTool;<br/>use Date::Parse;<br/>use POSIX;</pre></p><p>sub fixtime { return strftime("%Y:%m:%d %H:%M:%S", gmtime(str2time($_[0], "+0200"))); }</p><p>my $fname=$ARGV[0];<br/>if ($fname eq "") {<br/> print "usage: exif-cest2utc filename\n";<br/> exit(1);<br/>}</p><p>my $et = new Image::ExifTool;<br/>my $ef = $et->ExtractInfo($fname);</p><p>$et->SetNewValue("CreateDate", fixtime($et->GetValue("CreateDate", "Raw")));<br/>$et->SetNewValue("DateTimeOriginal", fixtime($et->GetValue("DateTimeOriginal", "Raw")));<br/>$et->WriteInfo($fname);</p>Roadtrip 2008: The GPS Track2008-10-16T12:11:00+00:00o'wolfhttps://woelfisch.de/blog/author/owolfhttps://woelfisch.de/blog/roadtrip-2008-the-gps-track<p>I've finally visualized the GPS recording of the RV Trip with my parents last Winter. Unfortunately, LJ does not allow me to embed the interactive map, thus you have to click on this static one to play with it:</p><p><a href="http://tracks.yaina.de/usa-2008-tracks.html"><img height="320" src="http://tracks.yaina.de/usa-2008-lj.png" width="480"/></a></p><p>Some facts about the map:</p><p><ul><li>4273354 GPS records logged in three weeks</li><li>930283 GPS positions logged</li><li>177717 GPS positions after eliminating errors, (nearly) identical positions and positions on straight lines</li><li>That's far too much for real-time rendering, be it client or server side.</li><li>Rendering the map at zoom level 18 took 28 minutes and 44 seconds and required 4 GB of memory</li><li>Zoom level 18 has 49895 tiles and 201 MB of PNG data</li><li>All zoom levels have 98682 tiles in total.</li><li>Selectable layers: OpenStreetMap Mapnik, Osmarender and NASA Global Mosaic</li></ul></p><p><strong>Edit:</strong> doesn't work with IE6, though.</p>