brapi_ping() now returns TRUE only when
/serverinfo answers with a successful JSON response, and
attaches the HTTP status as the "status" attribute.
Live-server tests skip with that status when the test server is down,
instead of failing (#42).
brapi_allele_matrix() and
brapi_get_dosage_matrix() now page
/allelematrix in both dimensions with the v2.1 parameters
dimensionVariantPage and dimensionCallSetPage.
They used to send a callSetPage parameter the specification
does not define, so a spec-compliant server repeated the first page of
call sets and later pages of variants were never fetched (#7).
brapi_post_search() and the
brapi_search_*() functions now return every page of a
search result, whether the server answers immediately or after an
asynchronous (HTTP 202) search. They used to return only the first
page_size records.
brapi_get_marker_map(variantSetDbId = ) no longer reports
variants as having no position when their positions were on a later page
(#8).
brapi_allele_matrix(),
brapi_get_dosage_matrix(), brapi_server_info()
and brapi_endpoints() now report HTTP errors with the
server’s own message, like every other function. A 401 used to come back
as an empty tibble, and an HTML error page as an httr2 content-type
error (#9).
brapi_get_dosage_matrix() now asks the server for
expanded homozygotes, its sep and its
unknown_string, and decodes calls more strictly. A call
with any missing allele ("./1") is NA rather
than a dosage, a collapsed homozygote ("1") is expanded
instead of becoming NA, and nucleotide calls
("A/T") are an error rather than being counted as alternate
alleles. The matrix is built directly from the allele matrix, without a
long-format pivot (#10).
Cached responses keep full numeric precision. They were written with jsonlite’s default of four decimal places, so a cached call could return different values from a fresh one; the parser’s fallback path rounded the same way. Cache entries are now written to a temporary file and renamed into place, so a parallel worker never reads a half-written entry (#11).
The cache key now includes a hash of the connection’s token, so a
response fetched with one token is never served to a connection with a
different token or none. brapi_cache_clear() removes only
brapiR2’s own cache files rather than everything in the directory, and
its examples no longer point the cache at tempdir()
(#12).
brapi_cache_enable() now defaults to
tools::R_user_dir("brapiR2", "cache"), R’s standard
per-user cache location, instead of
rappdirs::user_cache_dir(), and rappdirs is no
longer a suggested package. The test for the default directory no longer
writes to the user’s home directory (#13).
brapi_study_data() no longer drops repeated
measurements that happen to share a value: duplicates are now identified
by observationDbId. It also no longer fetches every
observation on the server when the filtered request comes back empty,
which is the normal answer for a study with no data. That fallback, for
servers that ignore the studyDbId filter such as the public
BrAPI test server, is now opt-in with fallback = TRUE, and
the empty-study message says so. Code that relied on the fallback needs
to set it (#14).
brapi_login_oauth2() now sends its
client-credentials token request form-encoded, as RFC 6749 requires,
rather than as JSON, which most OAuth servers reject. The unused
authorize_url argument is optional and deprecated;
supplying it gives a warning. The docs and README no longer describe an
authorization-code flow or imply EBS has been tested, and the example no
longer posts to the test server’s legacy /token endpoint,
which is not an OAuth server (#15).
Paginated requests now work out the page count from
totalCount and pageSize when a server omits
totalPages, instead of stopping after the first page. When
a server ignores the page parameter and serves the same
page again, brapi_get() stops with a warning instead of
repeating those records for every remaining page, and doesn’t cache the
result. max_pages is validated (#16).
A field whose JSON type differs between records, such as a plot
number that is 1 in one and "2A" in another,
no longer changes the shape of the result. The parser used to fall back
to jsonlite::fromJSON(flatten = TRUE), which turned nested
objects into dotted columns (geo became
geo.type); now such a column becomes character, or a
list-column if any record holds a list (#17).
Arguments are checked before any request is made. By-ID functions
such as brapi_study() need a single non-empty ID (a whole
number is accepted) and percent-encode it, so an ID with a space or
? no longer breaks the URL; NULL used to
request the API root and a vector a nonsense path. An ID containing
/ is refused, since it can’t be sent in a path.
NULL filters passed through ... are dropped, a
vector filter gives an error naming the parameter instead of an internal
one, brapi_connection() validates token,
version, page_size and timeout,
and brapi_study_data() says when records have no value
field. HTTP errors now name the server as well as the endpoint
(#18).
Logical query parameters, such as
includeParents = TRUE, are now sent as
true/false rather than
TRUE/FALSE. In search request bodies only the
plural array filters (germplasmDbIds, …) are sent as JSON
arrays; string fields the specification defines as single values, such
as sepUnphased, unknownString and
observationTimeStampRangeStart, are no longer wrapped in an
array (#19).
brapi_ping() uses a timeout of at most 10 seconds
and doesn’t retry, so an unreachable server is reported in under a
second instead of after retries (about 5 seconds per call with httr2
1.0.0, and every example pings first). A server that answers
/serverinfo with 401 or 403 is now reported as reachable
but needing authentication, rather than as down (#20).
brapi_fetch_parallel() now records which IDs failed:
the result carries a "failed" attribute listing each ID and
its error, one warning names them, and the summary says how many of the
IDs succeeded instead of claiming every source was fetched. Results
whose column types differ between IDs are combined (as character, or a
list-column) instead of aborting the whole batch (#21).
brapi_allele_matrix() and
brapi_get_dosage_matrix() no longer shift genotype calls
when the server sends a JSON null for a missing call: the
null becomes NA in its own cell instead of being dropped,
which moved every later call in that row to the wrong call set
(#67).
The README installation section now leads with CRAN, keeping
R-universe and GitHub as alternatives, and adds CRAN and R-universe
badges. ?brapi_get now counts 37 BrAPI v2.1 entities, and
?brapi_pedigree and DESIGN.md no longer say
brapi_germplasm_pedigree() calls the
/germplasm/{germplasmDbId}/pedigree sub-resource
(#6).
The README, vignettes and DESIGN.md no longer overstate what
brapiR2 returns or how it was tested: they name the functions that don’t
return a tibble, describe the server survey as one request per endpoint,
use the test server’s real IDs (study1,
variantset1) in the quick start, and no longer credit
untested servers with login styles (#22).
brapi_germplasm_pedigree(),
brapi_germplasm_progeny(),
brapi_server_info(), brapi_endpoints() and
brapi_ping() now take ..., so the query
parameters their help pages list can be passed. The help pages for
brapi_study_data(), brapi_get_dosage_matrix(),
brapi_get_marker_map(), brapi_login() and
brapi_login_oauth2() now name the endpoints they call, and
?brapi_login says /token is not part of BrAPI
v2.1 (#23).
Release housekeeping for 0.2.1: the NEWS headings use
## throughout, cran-comments.md describes an update rather
than a new release, CONTRIBUTING links the rOpenSci Code of Conduct and
lists only the test dependencies the tests use, and
dev/SERVERS.md notes that max_pages now
fetches a single page (#24).
CONTRIBUTING now describes how the live-server tests really skip:
on CRAN, and whenever the test server doesn’t answer
/serverinfo with HTTP 200 and JSON, rather than via
skip_if_offline() (#68).
New tests cover the error and paging paths that had none: a Breedbase 401 on a list endpoint, plain-text and HTML error bodies, a login that fails with the server’s reason, and an asynchronous search that is still running on more than one poll. With the tests added alongside the fixes for #7, #8, #11, #12, #16 and #42, every path listed in the issue is now tested (#26).
Removed unused imports (rlang::abort(),
inform(), warn(),
dplyr::select(), filter(),
purrr::map_dfr(), map_chr(),
map_lgl(), tidyr::unnest() and the
cli_progress_*() functions) and the unused httptest2 from
Suggests, fixed the remaining indentation and long-line lints, and added
missing words to inst/WORDLIST (#25).
The .Renviron example in the main vignette uses a
~~~ fence instead of a bare ``` one, which
goodpractice miscounted as an unmatched R chunk end, failing pkgcheck
with “Chunk start/end indices failed sanity checks in vignette”
(#42).
CI now checks the minimum supported R version (4.1), reports test coverage to Codecov, and runs R CMD check weekly so the live tests notice changes on the test server (#27).
The by-ID URL-encoding test no longer expects &
to be percent-encoded in a URL path. It checks that the ID’s path
segment has no raw space, ? or / and decodes
back to the ID, so it passes whichever way curl treats
& (#72).
brapi_get_marker_map() no longer reads position data
from brapi_variants()
(referenceName/start, which many servers -
including the public test server - leave NA, since the
BrAPI spec makes those fields optional on Variant). It now
queries the Genome Maps entity’s /markerpositions endpoint
instead, which places a marker on a named map (genetic or physical)
rather than a variant on a reference assembly. Signature
change:
brapi_get_marker_map(con, variantSetDbId = NULL, mapDbId = NULL),
requiring exactly one of the two identifiers. The returned tibble’s
columns have changed to variantDbId,
variantName, mapDbId, mapName,
type, unit, linkageGroupName,
position - referenceName/start
are gone. Existing positional calls
(brapi_get_marker_map(con, variantSetDbId)) still work, but
code reading referenceName or start from the
result will break.brapi_germplasm_pedigree() and
brapi_germplasm_progeny() now query
/pedigree?germplasmDbId= instead of the
/germplasm/{germplasmDbId}/pedigree and
/germplasm/{germplasmDbId}/progeny sub-resources, which
BrAPI deprecated in v2.1. The returned columns have
changed: 15 rather than 8, pedigree is now
pedigreeString, and parents,
siblings and progeny are list-columns of tidy
tibbles rather than raw nested lists. The added fields are
progeny, germplasmPUI,
defaultDisplayName, breedingMethodName,
breedingMethodDbId, additionalInfo and
externalReferences (@dwaring87,
ropensci/software-review#792).brapi_study_data() no longer silently returns the wrong
study’s observations on servers that don’t implement the
studyDbId filter on /observations server-side
(which triggers a client-side fallback that fetches all observations and
filters locally). The filter used
dplyr::filter(.data$studyDbId == studyDbId), which -
because the function’s own argument is also named studyDbId
- resolved the right-hand side to the data column itself, making the
comparison always TRUE and returning every study’s
observations rather than just the requested one. Fixed to
.data$studyDbId == .env$studyDbId, which correctly
disambiguates the data column from the function argument.brapi_study_data() no longer errors on studies where a
trait is measured on only some observation units.
pivot_wider() fills the absent combinations with
zero-length elements, and the simplification step’s
unlist() silently dropped them, returning a column shorter
than the table and failing inside dplyr::across().
Unmeasured cells are now NA, and simplified columns are
character throughout, which is how BrAPI returns observation values
(@dwaring87,
ropensci/software-review#792).brapi_study_data() additionally drops exact duplicate
records before pivoting, reporting how many it removed (@dwaring87,
ropensci/software-review#792).brapi_login() now builds its request the same way every
other function does. It previously hardcoded /brapi/, so
authentication was impossible on servers using a different path, and it
sent neither the user agent nor the Accept header.con$timeout is now applied to requests. The argument
has been accepted and documented since 0.1.0 but never took effect,
leaving every request on curl’s own default.brapi_study_data(), so values such as
"80 " do not become NA on conversion to
numeric. Reported by @dwaring87 from a development server; not
reproducible across 21 studies on T3/Oat Sandbox, T3/Wheat Sandbox and
Cassavabase, so the trim is defensive
(ropensci/software-review#792).brapi_germplasm_pedigree() and
brapi_germplasm_progeny() filter the response on
germplasmDbId client-side. The public test server ignores
that parameter on /pedigree, so a request for one germplasm
returned another alongside it (@dwaring87,
ropensci/software-review#792).result carries a field
named data is no longer mistaken for a collection. The
parser treated any result$data as the record envelope, so
/lists/{listDbId} returned only its members and discarded
every other field. data is now treated as the envelope only
when it is absent of scalars — that is, empty or holding objects./commoncropnames returns its crop
names that way, and the whole response came back as one row with a
list-column.HTTP 401 Unauthorized, and a failed login reports the
server’s reason — “Incorrect Password”, or “JSON array body required” —
instead of “no access token returned”. Servers report errors in several
different ways, and all of the forms seen across the test server,
Breedbase, T3 and GRIN-Global are handled; an HTML error page is not
shown, since it is never useful to a user (@dwaring87,
ropensci/software-review#792).brapi_study_data() checks that the observations it
received are really from the study asked for. A server may accept the
studyDbId filter and ignore it — the public test server
does exactly this on /pedigree — in which case every
study’s observations would have been pivoted together. Extra studies are
now filtered out, and the function says how many it found (@dwaring87,
ropensci/software-review#792).R/genome_maps.R):
brapi_maps(), brapi_map(),
brapi_map_linkage_groups(),
brapi_marker_positions(),
brapi_search_marker_positions().R/germplasm.R):
brapi_pedigree() and brapi_search_pedigree()
retrieve pedigree records across many germplasm in one call, via
/pedigree and /search/pedigree.
brapi_germplasm_pedigree() and
brapi_germplasm_progeny() now delegate to
brapi_pedigree() rather than calling the sub-resources
BrAPI deprecated in v2.1. Each row is one pedigree node;
parents, siblings and progeny,
when requested, are list-columns of tidy per-node tibbles rather than
raw nested lists, so a node with several relatives is never silently
collapsed to one row.R/phenotyping.R):
brapi_ontologies() and brapi_ontology(),
cross-referenced from brapi_traits(),
brapi_scales(), brapi_methods(), and
brapi_observation_variables().?brapi_coverage for the full coverage breakdown and which
entities remain uncovered.brapi_connection() gains a path argument
for servers that do not serve BrAPI under /brapi/.
GRIN-Global instances use gringlobal/brapi and were
previously unreachable; Germinate and GIGWA deployments commonly sit
under their own prefixes too. Defaults to "brapi", so
existing code is unaffected. print() shows the path only
when it differs from the default, and the cache key now includes it, so
two servers sharing a hostname no longer collide.brapi_connection() gains a user_agent
argument to override it (@dwaring87,
ropensci/software-review#792).brapi_location() retrieves a single location by ID,
so a user who knows a study’s locationDbId can fetch its
coordinates without listing every location and filtering. Verified
against the public test server, Cassavabase, T3/Oat Sandbox and
USDA-GRIN (@dwaring87,
ropensci/software-review#792).brapi_list() retrieves a single list by ID together
with its contents. brapi_lists() returns only metadata, so
there was previously no way to reach a list’s members at all. The
members come back as a character vector in the data
list-column, ready to pass to another function (@dwaring87,
ropensci/software-review#792).brapi_get() and brapi_post_search() are
now exported, so an endpoint brapiR2 does not wrap, a server extension,
or a query parameter no named function exposes can be reached without
dropping to raw HTTP. Pagination, caching, authentication and error
reporting work as they do for the named functions. Recommended by both
reviewers and by the rOpenSci packaging guidelines (@dwaring87, @jmh579,
ropensci/software-review#792).brapi_get() gains a max_pages argument. An
unfiltered call against a production server walks every page:
Cassavabase holds 8,539 studies and took 18 minutes to return them, and
a germplasm listing on T3/Wheat ran for over an hour before being
interrupted. There was no way to ask for just the first page. The
default fetches everything as before, and a truncated result is never
cached.brapi_fetch_parallel() no longer sets or restores a
future plan itself. Per the future package’s best-practices
vignette, the parallel backend is now the caller’s choice: call
future::plan() before calling
brapi_fetch_parallel() to fetch in parallel. The
.workers argument is deprecated - supplying it now emits a
warning and has no effect.LICENSE.md with the full MIT licence text, which
was missing from the repository, so GitHub had no licence to detect and
anyone opening LICENSE found no grant of rights. The
copyright holder is now named explicitly rather than “brapiR2 authors”
(@dwaring87,
ropensci/software-review#792).DESCRIPTION with the rev role for their
rOpenSci reviews (ropensci/software-review#792).future package at length. The
design rationale for why brapi_fetch_parallel() does not
set a plan has moved to DESIGN.md (@jmh579, ropensci/software-review#792).furrr and future being installed, so the
vignette builds where suggested packages are absent.DESCRIPTION now declares Language: en-GB,
and the package’s prose has been made consistent with it. Six American
spellings were corrected, and inst/WORDLIST has been
extended with the domain vocabulary and package names the spellchecker
cannot know (@jmh579,
ropensci/software-review#792).pak::pak() rather than
remotes::install_github(), which now warns (@jmh579,
ropensci/software-review#792), and notes that remotes and
devtools need build_vignettes = TRUE for the
vignette to be installed at all (@dwaring87,
ropensci/software-review#792).brapi_studies() documented only
trialDbId while the specification defines nine; the same
gap existed across the package (@dwaring87, @jmh579, ropensci/software-review#792).dev/brapi-spec.R reads the pinned
V2.1 tag of the BrAPI specification repository and writes
the endpoint inventory that the documentation is generated from.
Correcting against it: the specification defines 37 top-level entities
rather than 36, brapiR2 wraps 49 retrieval endpoints rather than 56, and
Planned Crosses was missing from the list of uncovered entities.?brapi_coverage lists every BrAPI endpoint brapiR2
wraps, grouped by module, with the function that wraps each one, and
names the entities that have no wrapper. Asked for by @jmh579 and @dwaring87
(ropensci/software-review#792).DESCRIPTION no longer names specific BrAPI
implementations. It claimed compatibility with Breedbase, BMS, EBS,
GIGWA and Germinate, of which only Breedbase had been tested; it now
claims the specification instead. The README records which servers
brapiR2 has actually been exercised against — eight, across three
implementations — and which it has not (@jmh579, ropensci/software-review#792).DESIGN.md no longer lists four packages of mine as
brapiR2’s downstream pipeline. Two are unreleased and two are at early
versions, so the pipeline was described as though it were
established.saveRDS() and printed by str(), while
print() shows only whether the connection is
authenticated.\donttest{} examples on
some check flavours.path, the
user agent as sent and as overridden, login reaching a non-standard
path, JSON key-order normalisation, unmeasured traits filling with
NA rather than shortening the column, whitespace trimming,
the client-side studyDbId and germplasmDbId
filters, the three response shapes the parser must tell apart, and the
error-message extraction for each form a server uses.brapi_programs(),
brapi_trials(), brapi_studies(),
brapi_locations(), brapi_seasons(),
brapi_lists(), brapi_people(),
brapi_server_info().brapi_germplasm(),
brapi_germplasm_pedigree(),
brapi_germplasm_progeny(), brapi_crosses(),
brapi_crossing_projects(), brapi_seed_lots(),
brapi_search_germplasm().brapi_observation_units(),
brapi_observations(),
brapi_observation_variables(), brapi_traits(),
brapi_scales(), brapi_methods(),
brapi_images(), brapi_events(),
brapi_search_observations(),
brapi_search_variables().brapi_samples(),
brapi_variants(), brapi_variant_sets(),
brapi_calls(), brapi_call_sets(),
brapi_references(), brapi_reference_sets(),
brapi_allele_matrix(),
brapi_search_variants(),
brapi_search_calls().brapi_study_data() (wide-format
phenotype table), brapi_get_dosage_matrix(),
brapi_get_marker_map().brapi_connection() — no global state or side
effects.brapi_cache_enable().brapi_fetch_parallel().