Marian Steinbach: Blog

Japan Radiation Open Data

Measurement Data

Download the latest radiation measurement data (“space dose rate”) dump as a gzipped CSV file:

station_data_1h.csv.gz (recommended): contains only the values of the last hour. Recommended for repeated downloads. GZip compressed to save bandwidth. Updated every ten minutes.

station_data_1h.csv: Same as if above, but uncompressed. In case you have trouble with the compression. Updated every ten minutes.

station_data.csv.gz is the ever growing complete data download starting March 1, 2011. As of November 2011, this file contains over 7 Mio lines. Please only download this if you need historical data. Updated once per day.

Please make sure to avoid redundant downloads, resulting in unnecessary bandwidth usage.

Format

The files contain one measurement per row, indicating the date/time of the measurement, the station and the value measured. Fields are separated with tabs. The first row contains column names.

The colums are:

  • datetime_utc: The UTC time when the data has been measured. To convert from UTC to Japan local time (JST), add 9 hours.
  • station_id: Unique id of the station, e.g. 1010000001. See section “Station Location Data” for details.
  • sa: The measured radiation dose value in the unit nGy/h, which is nano Gray per hour. (Sometimes the value -888 or -999 appears. I’m not sure what this means, but these are the values published by the source website.) For information on how to interpret the nGy/h values, please refer to this table. For perspective, the average value before the accident at Fukushima Dai-ichi (timespan March 1 to March 12, 2011) over all stations was around 39 with a standard deviation of 15.
  • ra: precipitation (rain) in mm. Same for -888 and -999

Please note that the schema might change in the future.

Station Location Data

In the measurement data above, stations are referenced by numeric IDs only. The complete table of stations, including ID, geo location and names in Japanese and English language are available in two sources:

Data Status

as of 2011-08-28 08:20 UTC

  • Data values from 2011-08-09 to 2011-08-14 are missing due to failure on my end. I will work on retrieving the missing values to fill the gap. As soon as this is accomplished, you will find the according notice right here.
  • The old download format (http://s.sendung.de/jpradiation/data.csv.gz) has been disabled. Please migrate to the station_data_1h.csv.gz format to save your and my resources.
  • Schema change: The download station_data.csv.gz only contains Station ID, Date/Time and radiation information (and precipitation). See section “Station Location Data” on this page for information on station locations.
  • The prefecture and station names are not contained in the new download. They can be accessed in form of a Google Docs Spreadsheet. The station_id is the key to relate both sources to each other.
  • Values now reach back until March 1, 2011!

Background

People around the world are concerned about the level of radiation in Japan due to major damages on the Fukushima Daiishi nuclear power plant. The Ministry of Education, Culture, Sports, Science and Technology (MEXT) of Japan publishes real time radiation measurement data acquired by the “System for Prediction of Environment Emergency Dose Information” (SPEEDI).

However, these measurements published by the japanese administration are not very accessible to the public. Raw data cannot be accessed. And historic values are only available as graphical representations.

In order to make data of public interest accessible to everybody, I started a Google Docs spreadsheet and called out to the online world for help with gathering the values there. It was a mixed success. Lots of great people have worked on this spreadsheet and contributed time to manually enter data in order to give others access to the radiation data. Additional sources like private Geiger counters have been tracked. But also trolls discovered the spreadsheet as their playground.

The next step then was to harvest the published data programmatically. Measurements are now crawled automatically and extracted from the HTML pages. The extracted data is provide in the CSV file available above.

Applications and derivatives of the data

  • Map view plus time series graphs by Geir Endahl
  • Graph Dashboard by Eron Villarreal
  • pachube.com gets the updates for each stations as soon as they arrive, so you can use pachube’s data feeds to grab the data in various formats. Search for radiation japan webscrape.
  • RDTN.org safecast.org uses the data for a map visualization, (potentially) together with other data from pachube.com.
  • japan.failedrobot.com maps the current values via the pachube.com feeds. Values are converted to µSv (micro Sievert).
  • radiocial.org is similar to RDTN.ORG and directly integrates the data feed.
  • Graph Dashboard by myself. Gives you an aggregated view per prefection, either per hour over the last 72 hours or per day for the complete time frame contained in the data
  • I have started to create simple Animations of the data.
  • Rama Hoetzlein has a one-page infographic containing spatial and time-series information consisting of various sources.

Please let me know of more uses you know about.

Discuss

Please comment in context of the according blog post or contact me via twitter @MarianSteinbach.