Help for package clidatajp

Title:

Data from Japan Meteorological Agency

Version:

0.5.3

Description:

Includes climate data from Japan Meteorological Agency ('JMA') https://www.jma.go.jp/jma/indexe.html. Can download climate data from 'JMA'.

License:

MIT + file LICENSE

Encoding:

UTF-8

Depends:

R (≥ 3.5.0)

URL:

https://github.com/matutosi/clidatajp https://github.com/matutosi/clidatajp/tree/develop (devel) https://matutosi.github.io/clidatajp/

LazyData:

true

Imports:

curl, dplyr, httr, magrittr, rlang, rvest, stats, stringi, stringr, tibble, tidyr, utils

Suggests:

ggplot2, knitr, rmarkdown, spelling, testthat (≥ 3.0.0)

VignetteBuilder:

knitr

Config/testthat/edition:

Language:

en-US

Config/roxygen2/version:

8.1.0

NeedsCompilation:

Packaged:

2026-08-26 21:36:26 UTC; matutosi

Author:

Toshikazu Matsumura [aut, cre]

Maintainer:

Toshikazu Matsumura <matutosi@gmail.com>

Repository:

CRAN

Date/Publication:

2026-08-26 22:10:02 UTC

Pipe operator

Description

See magrittr::%>% for details.

Usage

lhs %>% rhs

Arguments

lhs

A value or the magrittr placeholder.

rhs

A function call using the magrittr semantics.

Value

The result of calling rhs(lhs).

Wrapper function to convert into numeric without warnings

Description

Wrapper function to convert into numeric without warnings

Usage

as_numeric_without_warnings(x)

Arguments

x

A string.

Value

A numeric or NA.

Choose data with menu.

Description

Choose data with menu.

Usage

choose_with_menu(df, filter_cols, extract = NULL)

Arguments

df

A dataframe

filter_cols

A string or string vector

extract

A string

Value

If extract is NULL, return a dataframe, else return a vector.

Examples

if(interactive()){
  data(climate_world)
  climate_world <- 
    climate_world %>%
    dplyr::mutate_all(stringi::stri_unescape_unicode)
  
  choose_with_menu(climate_world, filter_cols = "continent")
  4  # input
  
  choose_with_menu(climate_world, filter_cols = c("continent", "country", "station"))
  4  # input
  3  # input
  2  # input
}

Clean up station information

Description

Helper function for download_climate().

Usage

clean_station(station)

Arguments

station

A String of station information.

Value

A tibble including station information.

Examples

data(station_links)
station_links %>%
  head(1) %>%
  `$`("station") %>%
  stringi::stri_unescape_unicode() %>%
  clean_station()

Climate data in Japan

Description

Climate data downloaded from Japan Meteorological Agency web pages. URLs of each station are listed in data(station_links). https://www.data.jma.go.jp/gmd/cpd/monitor/nrmlist/

Usage

climate_jp

japan_climate

Format

A data frame with 3768 (157 stations * 12 months * 2 periods) rows and 14 variable:

no: Station no
month: Month
period: Period of observations
temperature: Mean temperature
precipitation: Mean precipitation
snowfall: Mean snowfall
insolation: Mean insolation
station: Station name. To avoid duplication, including country name after station name. Can split by "_". Escaped by stringi::stri_escape_unicode().
country: Country name. Escaped by stringi::stri_escape_unicode().
latitude: Latitude. (degree)
NS: North or South.
longitude: Longitude. (degree)
WE: West or East.
altitude: Altitude (m)

An object of class tbl_df (inherits from tbl, data.frame) with 3768 rows and 14 columns.

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(japan_climate)
japan_climate %>%
  dplyr::mutate_all(stringi::stri_unescape_unicode)

Climate normals of all stations in Japan

Description

Monthly normals downloaded from Japan Meteorological Agency web pages for all stations in data(station_jp_full). Values are raw strings of the web pages, because they include marks of 'JMA' ("@", ")", "]" and "#" show quality of the value, "///" and "—" show no value). Use as_numeric_without_warnings() after removing the marks, or use download_detail(), which cleans the marks. Stations of "AMeDAS" have no value in 11 of the columns, because only 10 items are observed. https://www.data.jma.go.jp/obd/stats/etrn/index.php

Usage

climate_jp_full

Format

A data frame with 21697 rows (1669 stations * 13 rows) and 21 variables. All variables are character.

month: Month (1 to 12) or yearly value. Escaped by stringi::stri_escape_unicode().
air_pressure_land: Mean air pressure on land (hPa)
air_pressure_sea: Mean air pressure on sea level (hPa)
precipitation: Mean precipitation (mm)
temperature: Mean temperature (degree Celsius)
temperature_max: Mean of daily maximum temperature (degree Celsius)
temperature_min: Mean of daily minimum temperature (degree Celsius)
steam_pressure: Mean steam pressure (hPa)
moisture: Mean relative humidity (percent)
wind_speed: Mean wind speed (m/s)
wind_direction: Most frequent wind direction. Escaped by stringi::stri_escape_unicode().
sunshine: Mean sunshine duration (hour)
global_solar_radiation: Mean global solar radiation (MJ/m^2)
snow_fall: Mean snow fall (cm)
snow_fall_max_per_day: Mean of daily maximum snow fall (cm)
snow_deepest: Mean of deepest snow (cm)
cloud_cover: Mean cloud cover
days_of_snow: Mean days of snow
days_of_fog: Mean days of fog
days_of_thunderstorm: Mean days of thunderstorm
station_no: Station no. Can be joined to station_no of data(station_jp_full). Escaped by stringi::stri_escape_unicode().

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(climate_jp_full)
climate_jp_full %>%
  head_3() %>%
  dplyr::mutate_if(is.character, stringi::stri_unescape_unicode)

Climate data in the world

Description

Climate data downloaded from Japan Meteorological Agency web pages. URLs of each station are listed in data(station_links). https://www.data.jma.go.jp/gmd/cpd/monitor/nrmlist/

Usage

climate_world

world_climate

Format

A data frame with 41328 (3444 stations * 12 months) rows and 12 variable:

no: Station no
continent: Continent. Escaped by stringi::stri_escape_unicode().
country: Country name. Escaped by stringi::stri_escape_unicode().
station: Station name. To avoid duplication, including country name after station name. Can split by "_". Escaped by stringi::stri_escape_unicode().
month: Month
temperature: Mean temperature
precipitation: Mean precipitation
latitude: Latitude. (degree)
NS: North or South.
longitude: Longitude. (degree)
WE: West or East.
altitude: Altitude (m)

An object of class tbl_df (inherits from tbl, data.frame) with 41328 rows and 12 columns.

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(world_climate)
world_climate %>%
  dplyr::mutate_all(stringi::stri_unescape_unicode)

Helper functions for download_detail()

Description

detail_colnames() joins header rows into column names, clean_detail_value() removes marks of 'JMA' from values, and to_numeric_when_all() converts a vector into numeric only when all values can be converted.

Usage

detail_colnames(header)

clean_detail_value(x)

to_numeric_when_all(x)

Arguments

header

A dataframe of header rows.

x

A string vector of values.

Value

detail_colnames(): a string vector of column names. clean_detail_value(): a string vector. to_numeric_when_all(): a numeric vector or a string vector.

Examples

header <- data.frame(x = c("temperature", "temperature"),
                     y = c("temperature", "mean"))
detail_colnames(header)
clean_detail_value(c("21.6", "0.0 )", "--", "///"))
to_numeric_when_all(c("21.6", "0.0", NA))
to_numeric_when_all(c("21.6", "north"))

Build urls of detail climate data in Japan

Description

Japan Meteorological Agency ('JMA') provides detail climate data (yearly, monthly, daily, hourly and 10 minutes values) for each station. detail_url() builds urls of the data, and download_detail() downloads the data of the url. You can see web page as below. https://www.data.jma.go.jp/obd/stats/etrn/index.php

Usage

detail_url(
  prec_no,
  block_no,
  item = "daily",
  year = NULL,
  month = NULL,
  day = NULL
)

detail_item(item, block_no)

Arguments

prec_no

A string or numeric of area no.

block_no

A string or numeric of station no. Keep leading zero of 'AMeDAS' as a string (ex. "0588").

item

A string to specify data type. See details.

year

A numeric or string of year.

month

A numeric or string of month.

day

A numeric or string of day.

Details

A station is specified by a pair of prec_no (area) and block_no (station). Use download_prec_no() and download_block_no() to get the numbers. Note that block_no of "kansho" (weather station) has 5 digits (ex. "47759") and that of "AMeDAS" has 4 digits (ex. "0588"). Suffix of item ("_s" or "_a") is set to match the type of block_no.

item can be a short name in the following list, or a name of php file without extension (ex. "daily_s1").

annually: Yearly values.
monthly: Monthly values in a year (year is required).
monthly_all: Monthly values of all years ("kansho" only).
daily: Daily values in a month (year and month are required).
hourly: Hourly values in a day (year, month and day are required).
10min: 10 minutes values in a day (year, month and day are required).
nml_ym: Normals of each month.
nml_daily: Normals of each day in a month (month is required).
nml_season: Normals of first and last day of frost, snow and ice ("kansho" only).

Value

    A string vector of urls.

Examples

detail_url(61, 47759, "daily", 2023, 6)
detail_url(61, "0588", "hourly", 2023, 6, 20)
detail_url(61, 47759, "daily", 2023, 1:3)

Download climate data of the world

Description

For polite scraping, 5 sec interval is set in download_climate(), it takes over 5 hours to get climate data of all stations. Please use existing links by "data(climate_world)", if you do not need to renew climate data. You can see web page as below. https://www.data.jma.go.jp/gmd/cpd/monitor/nrmlist/

Usage

download_climate(url)

Arguments

url

A String to specify target html.

Value

A tibble including climate and station information, or NULL when failed.

Examples

# If you want all climate data, remove head().
# The codes take > 5 sec because of poliste scraping.

library(magrittr)
library(stringi)
library(dplyr)
data(station_links)
station_links <-
  station_links %>%
  dplyr::mutate_all(stringi::stri_unescape_unicode) %>%
  head(3) %T>%
  { 
     continent <<- `$`(., "continent") 
     no        <<- `$`(., "no") 
  } %>%
  `$`("url")

climate <- list()
for(i in seq_along(station_links)){
  print(stringr::str_c(i, " / ", length(station_links)))
  climate[[i]] <- download_climate(station_links[i])
}
  # run only when download_climate() successed
if(sum(is.null(climate[[1]]), 
       is.null(climate[[2]]), 
       is.null(climate[[3]])) == 0){
  month_per_year <- 12
  climate_world <- 
    dplyr::bind_rows(climate) %>%
    dplyr::bind_cols(
      tibble::tibble(continent = rep(continent, month_per_year))) %>%
    dplyr::bind_cols(
      tibble::tibble(no        = rep(no,        month_per_year))) %>%
    dplyr::relocate(no, continent, country, station)
  climate_world
}

Download detail climate data in Japan

Description

For polite scraping, 5 sec interval is set in download_detail(). Please do not download too many data at once. Use detail_url() to build an url of the data.

Usage

download_detail(url, as_numeric = TRUE)

Arguments

url

A string to specify target html.

as_numeric

A logical. If TRUE, columns of numbers are converted into numeric. Marks of 'JMA' for missing values are treated as NA.

Value

A tibble including detail climate data, or NULL when failed.

Examples


url <- detail_url(61, 47759, "daily", 2023, 6)
download_detail(url)

Download links for areas, countries and stations

Description

For polite scraping, 5 sec interval is set in download_links(), it takes about 15 minutes to get all station links. Please use existing links by "data(station_links)", if you do not need to renew links. You can see web page as below. https://www.data.jma.go.jp/gmd/cpd/monitor/nrmlist/

Usage

download_area_links(
  url = "https://www.data.jma.go.jp/gmd/cpd/monitor/nrmlist/"
)

download_links(url)

Arguments

url

A String to specify target html.

Value

A string vector of url links, or NULL when failed.

Examples

# If you want links for all countries and all sations, remove head().
# The codes take over 5 sec because of poliste scraping.

library(magrittr)
library(stringi)
library(dplyr)
library(tibble)
area_links <- download_area_links()
station_links <- NULL
continent     <- NULL
continents <- 
  c("\\u30a2\\u30d5\\u30ea\\u30ab",
    "\\u30a2\\u30b8\\u30a2",
    "\\u5357\\u30a2\\u30e1\\u30ea\\u30ab", 
    "\\u5317\\u4e2d\\u30a2\\u30e1\\u30ea\\u30ab",
    "\\u30aa\\u30bb\\u30a2\\u30cb\\u30a2",
    "\\u30e8\\u30fc\\u30ed\\u30c3\\u30d1")
area_links <- head(area_links, 1)  # for test
for(i in seq_along(area_links)){
    print(stringr::str_c("area: ", i, " / ", length(area_links)))
    country_links <- download_links(area_links[i])
    country_links <- head(country_links, 1)  # for test
    for(j in seq_along(country_links)){
        print(stringr::str_c("    country: ", j, " / ", length(country_links)))
        links <- download_links(country_links[j])
        station_links <- c(station_links, links)
        continent     <- c(continent,     rep(continents[i], length(links)))
    }
}
station_links <- tibble::tibble(url = station_links, continent = continent)
station_links

Download prec_no and block_no of stations in Japan

Description

A station of detail climate data is specified by a pair of prec_no (area) and block_no (station). download_prec_no() downloads all prec_no, and download_block_no() downloads block_no of the area. For polite scraping, 5 sec interval is set in both functions. You can see web page as below. https://www.data.jma.go.jp/obd/stats/etrn/index.php

Usage

download_prec_no()

download_block_no(prec_no)

Arguments

prec_no

A string or numeric of area no.

Value

download_prec_no(): a tibble of prec_no and area. download_block_no(): a tibble of prec_no, block_no, station and type. NULL when failed.

Examples


prec_no <- download_prec_no()
prec_no
download_block_no(61)

Graceful fail

Description

Graceful fail

Usage

gracefully_fail(remote_file)

Arguments

remote_file

A string of remote file.

Value

 An XML document when successed, or invisible NULL when failed.

References

https://gist.github.com/kvasilopoulos/47f24348ed75cdb6365312b17f4b914c

Wrapper function to head 3 items

Description

Wrapper function to head 3 items

Usage

head_3(x)

Arguments

x

An object.

Value

An object like x with length 3.

Downloaded tables of climate normals in Japan

Description

Raw tables downloaded from Japan Meteorological Agency web pages, before cleaning into data(climate_jp_full). Kept for reproducibility of data-raw/climate_jp_full.R. Each element is a table of a station of data(station_jp_full), and the order of the elements is the same as the rows of the data. Column names are not set, and the first rows are headers of the web page. https://www.data.jma.go.jp/obd/stats/etrn/index.php

Usage

mean_cli

Format

A list of 1673 tibbles.

21 columns: 157 tables of "kansho" (weather station).
10 columns: 1512 tables of "AMeDAS".
NULL: 4 stations, which have no normals.

Examples

data(mean_cli)
length(mean_cli)
dim(mean_cli[[1]])

Wrapper function to sleep

Description

Wrapper function to sleep

Usage

sleep(sec = 5)

Arguments

sec

A numeric to sleep (sec).

Value

No return value, called for side effects.

Climate stations in Japan

Description

Climate stations in Japan

Usage

station_jp

Format

A data frame with 157 rows and 11 variable:

region: Region. Escaped by stringi::stri_escape_unicode().
pref: Prefecture. Escaped by stringi::stri_escape_unicode()
no: Station no.
station: Station name. To avoid duplication, including country name after station name. Can split by "_". Escaped by stringi::stri_escape_unicode().
altitude: Altitude. (m)
latitude: Latitude. (degree)
longitude: Longitude. (degree)
NS: North or South.
WE: West or East.
yomi: Pronunciation in Japanese. Escaped by stringi::stri_escape_unicode()
city: City name. Escaped by stringi::stri_escape_unicode().

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(station_jp)
station_jp %>%
  dplyr::mutate_all(stringi::stri_unescape_unicode)

All stations in Japan

Description

Station information downloaded from Japan Meteorological Agency web pages. data(station_jp) includes only 157 stations of climate data, while this data includes all 1673 stations. A pair of prec_no and station_no specifies a station in detail_url() and download_detail(). Note that station_no of "kansho" (weather station) has 5 digits (159 stations) and that of "AMeDAS" has 4 digits (1514 stations). https://www.data.jma.go.jp/obd/stats/etrn/index.php

Usage

station_jp_full

Format

A data frame with 1673 rows and 16 variables:

station: Station name. Escaped by stringi::stri_escape_unicode().
prec_no: Area no of 'JMA'. Escaped by stringi::stri_escape_unicode().
station_no: Station no of 'JMA', which is used as block_no in detail_url(). Escaped by stringi::stri_escape_unicode().
yomi: Pronunciation in Japanese. Escaped by stringi::stri_escape_unicode().
altitude: Altitude (m)
precipitation: Flag of observation (1: observed, 0: not observed). Character, not numeric.
wind: Flag of observation (1: observed, 0: not observed)
temperature: Flag of observation (1: observed, 0: not observed)
sunshine: Flag of observation (1: observed, 0: not observed)
snow: Flag of observation (1: observed, 0: not observed)
moisture: Flag of observation (1: observed, 0: not observed)
prec: Area name of prec_no. Escaped by stringi::stri_escape_unicode().
latitude: Latitude (degree)
longitude: Longitude (degree)
end_date: Date of the end of observation. "9999-99-99" shows that the station is in operation (1287 stations). Escaped by stringi::stri_escape_unicode().
memo: Note of 'JMA'. Escaped by stringi::stri_escape_unicode().

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(station_jp_full)
station_jp_full %>%
  head_3() %>%
  dplyr::mutate_if(is.character, stringi::stri_unescape_unicode)

Station name and its URL

Description

Station name and its URL

Usage

station_links

Format

A data frame with 3444 rows and 4 variable:

no: Station no
station: Station information including no, month, temperature, precipitation, station, country, latitude, NS, longitude, WE, altitude. The information is NOT cleaned Row information downloaded from each URL. Escaped by stringi::stri_escape_unicode().
url: URL of station.
continent: Continent. Escaped by stringi::stri_escape_unicode().

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(station_links)
station_links %>%
  dplyr::mutate_all(stringi::stri_unescape_unicode)

Climate stations of the world

Description

Climate stations of the world

Usage

station_world

Format

A data frame with 3444 rows and 9 variable:

no: Station no
station: Station name. To avoid duplication, including country name after station name. Can split by "_". Escaped by stringi::stri_escape_unicode().
continent: Continent. Escaped by stringi::stri_escape_unicode().
country: Country name. Escaped by stringi::stri_escape_unicode().
altitude: Altitude (m)
latitude: Latitude (degree)
NS: North or South.
longitude: Longitude (degree)
WE: West or East

Examples

library(magrittr)
library(stringi)
library(dplyr)
data(station_world)
station_world %>%
  dplyr::mutate_all(stringi::stri_unescape_unicode)

Calculate warm index and cold index

Description

Calculate warm index and cold index

Usage

wi(x)

ci(x)

Arguments

x

A numeric vector

Value

A string vector of url links.

References

Kira, T. 1945. A new classification of climate in eastern Asia as the basis for agricultural geography, Hort. Inst. Kyoto Univ., Kyoto. (in Japanese) Warmth Index (WI) and Cold Index (CI) was proposed by Kira (1945), which is known closely related to the distribution of vegetation. Indices can are calculated by following equations. wi = sum (Ti - 5), where wi is Warm index, Ti (Celsius) is mean temperature of each month in a year when Ti > 5. Indices can are calculated by following equations. wi = -sum (Ti - 5), where wi is Cold index, when Ti < 5.

Examples

temp <- c(-7.8, -7.2, -2.4, 5.2, 11.7, 16.5, 20.5, 21.1, 15.6, 8.8, 2.0, -4.1)
wi(temp)
ci(temp)
wi <- sum(c(0, 0, 0, 0.2, 6.7, 11.5, 15.5, 16.1, 10.6, 3.8, 0, 0))
ci <- sum(c(12.8, 12.2, 7.4, 0, 0, 0, 0, 0, 0, 0, 3.0, 9.1))

Package {clidatajp}

Pipe operator

Description

Usage

Arguments

Value

Wrapper function to convert into numeric without warnings

Description

Usage

Arguments

Value

Choose data with menu.

Description

Usage

Arguments

Value

Examples

Clean up station information

Description

Usage

Arguments

Value

Examples

Climate data in Japan

Description

Usage

Format

Examples

Climate normals of all stations in Japan

Description

Usage

Format

Examples

Climate data in the world

Description

Usage

Format

Examples

Helper functions for download_detail()

Description

Usage

Arguments

Value

Examples

Build urls of detail climate data in Japan

Description

Usage

Arguments

Details

Value

Examples

Download climate data of the world

Description

Usage

Arguments

Value

Examples

Download detail climate data in Japan

Description

Usage

Arguments

Value

Examples

Download links for areas, countries and stations

Description

Usage

Arguments

Value

Examples

Download prec_no and block_no of stations in Japan

Description

Usage

Arguments

Value

Examples

Graceful fail

Description

Usage

Arguments

Value