The rtrek package includes some Star Trek datasets, but
much more data is available outside the package. You can access other
Star Trek data through various APIs.
Technically, there is only one formal API: the Star Trek API (STAPI).
rtrek has functions to assist with making calls to this API
in order to access specific data. This is the focus of this
vignette.
rtrek also interfaces with and extracts information from
the Memory
Alpha and Memory Beta
websites. Neither of these sites actually expose an API, but functions
in rtrek with querying these websites in an API-like
manner. See the respective vignettes for details.
Accessing information from STAPI is covered to some degree in the package introduction vignette. There is some duplication here, followed by additional examples.
Star Trek API
The Star Trek API (STAPI) is a particularly useful data source.
Keep in mind that STAPI focuses more on providing real world data associated with Star Trek (e.g., when did episode X first air on television?) than on fictional universe data, but it contains both and the database holdings will grow with time.
To use the words of the developers, the STAPI is
the first public Star Trek API, accessible via REST and SOAP. It’s an open source project, that anyone can contribute to.
The API is highly functional. Please do not abuse the API with
constant requests. Their pages suggest no more than one request per
second, but I would suggest ten seconds between successive requests. The
default anti-DDOS measures in rtrek limit requests to one
per second. You can update this global rtrek setting with
options, e.g. options(rtrek_antidos = 10) for
a minimum ten second wait between API calls to be an even better
neighbor. rtrek will not permit faster requests. If set
below one second, the option is ignored and a warning thrown when making
any API call.
STAPI entities
There a many fields, or entities, available in the API. The available IDs can be found in this table:
stapiEntities
#> # A tibble: 40 × 4
#> id class ncol colnames
#> <chr> <chr> <int> <named list>
#> 1 animal tbl_df 7 <chr [7]>
#> 2 astronomicalObject tbl_df 5 <chr [5]>
#> 3 book tbl_df 24 <chr [24]>
#> 4 bookCollection tbl_df 10 <chr [10]>
#> 5 bookSeries tbl_df 11 <chr [11]>
#> 6 character tbl_df 24 <chr [24]>
#> 7 comicCollection tbl_df 14 <chr [14]>
#> 8 comics tbl_df 15 <chr [15]>
#> 9 comicSeries tbl_df 15 <chr [15]>
#> 10 comicStrip tbl_df 12 <chr [12]>
#> # ℹ 30 more rowsThese ID values are passed to stapi to perform a search
using the API. The other columns provide some information about the
object returned from a search. All entity searches return tibble data
frames. You can inspect or unnest the column names of each table
returned from every available entity search so you can see beforehand
what variables are associated with each entity.
Accessing the API
Using stapi should be thought of as a three part
process:
- Determine how many pages of results exist for a particular entity search.
- Only after taking care to do the previous step, perform the search to return search results.
- If satellite data is needed on a unique observation in the search
results, call
stapione more time referencing the specific observation.
To determine how many pages of results exist for a given search, set
page_count = TRUE. The impact on the API will be equivalent
to only searching a single page of results. One page contains metadata
including the total number of pages. Nothing is returned in this “safe
mode”, but the total number of search results available is printed to
the console.
Searching movies only returns one page of results. However, there are a lot of characters in the Star Trek universe. Check the total pages available for character search.
stapi("character", page_count = TRUE)
#> Total pages to retrieve all results: 76And that is with 100 results per page!
The default page = 1 only returns the first page.
page can be a vector, e.g. page = 1:62.
Results from multi-page searches are automatically combined into a
single, constant data frame output. For the second call to
stapi, return only page two here, which contains the
character, Q (currently, pending future character database updates that
may shift the indexing). In case that does change and Q is not always
near the top of page two of the search results, the example further
below hard-codes his unique/universal ID.
library(dplyr)
stapi("character", page = 1) %>% select(uid, name)
#> # A tibble: 100 × 2
#> uid name
#> <chr> <chr>
#> 1 CHMA0000215045 0413 Theta
#> 2 CHMA0000174718 0718
#> 3 CHMA0000283851 10111
#> 4 CHMA0000278055 335
#> 5 CHMA0000282741 355
#> 6 CHMA0000026532 A'trom
#> 7 CHMA0000280385 A. Armaganian
#> 8 CHMA0000226457 A. Baiers
#> 9 CHMA0000232390 A. Baiers
#> 10 CHMA0000068580 A. Banda
#> # ℹ 90 more rowsCharacter tables can be sparse. There are a lot of variables, many of which will contain missing data for rare, esoteric characters. Even for more popular characters about whom much more universe lore has been uncovered, it still takes dedicated nerds to enter all the data in a database.
When a dataset contains a uid column, this can be used
subsequently to extract a satellite dataset about that particular
observation that was returned in the original search. First you used
safe mode, then search mode, and now switch from search mode to
extraction mode to obtain data about Q, specifically. All that is
required to do this is pass Q’s uid to stapi
and call the function one last time. When uid is no longer
NULL, stapi knows not to bother with a search
and makes a different type of API call requesting information about the
uniquely identified entry.
Q <- "CHMA0000025118"
Q <- stapi("character", uid = Q)
q_eps <- Q$episodes %>% select(uid, title, stardateFrom, stardateTo)
q_eps
#> uid title stardateFrom stardateTo
#> 1 EPMA0000259941 Veritas NA NA
#> 2 EPMA0000000651 Tapestry NA NA
#> 3 EPMA0000000500 Hide And Q 41590.5 41590.5
#> 4 EPMA0000277408 The Star Gazer NA NA
#> 5 EPMA0000280052 Farewell NA NA
#> 6 EPMA0000279099 Two of One NA NA
#> 7 EPMA0000278606 Watcher NA NA
#> 8 EPMA0000001510 The Q and the Grey 50384.2 50392.7
#> 9 EPMA0000001413 True Q 46192.3 46192.3
#> 10 EPMA0000000845 Q-Less 46531.2 46531.2
#> 11 EPMA0000001329 Q Who 42761.3 42761.3
#> 12 EPMA0000278900 Fly Me to the Moon NA NA
#> 13 EPMA0000000483 Encounter at Farpoint 41153.7 41153.7
#> 14 EPMA0000001458 All Good Things... 47988.0 47988.0
#> 15 EPMA0000162588 Death Wish 49301.2 49301.2
#> 16 EPMA0000289337 The Last Generation NA NA
#> 17 EPMA0000001347 Deja Q 43539.1 43539.1
#> 18 EPMA0000277535 Penance NA NA
#> 19 EPMA0000278226 Assimilation NA NA
#> 20 EPMA0000279450 Mercy NA NA
#> 21 EPMA0000001619 Q2 54704.5 54704.5
#> 22 EPMA0000001377 Qpid 44741.9 44741.9The data returned on Q is actually a large list, including multiple data frames. For simplicity only a piece of it is shown above.
Find out which TNG characters other than Q appear in both the
Encounter at Farpoint series premier and later in the All
Good Things… series finale. To do this, usestapi to
extract data from other endpoints by following a breadcrumb trail of
uid values.
Engage.
eps <- c("Encounter at Farpoint", "All Good Things...")
q_eps <- filter(q_eps, title %in% eps)
q_eps
#> uid title stardateFrom stardateTo
#> 1 EPMA0000000483 Encounter at Farpoint 41153.7 41153.7
#> 2 EPMA0000001458 All Good Things... 47988.0 47988.0
eaf <- stapi("episode", uid = q_eps$uid[q_eps$title == eps[1]])
agt <- stapi("episode", uid = q_eps$uid[q_eps$title == eps[2]])
characters <- setdiff(intersect(eaf$characters$name, agt$characters$name), "Q")
characters
#> [1] "Deanna Troi" "William T. Riker" "Miles O'Brien" "Natasha Yar"
#> [5] "Worf" "Geordi La Forge" "Data" "Jean-Luc Picard"
#> [9] "Beverly Crusher"This returns key crew members who remained a part of the show from
beginning to end, disregarding any interim absences. Below, inspect how
many episodes each character appeared in. uid is again
needed, this time for each character.
Note that this requires making one API call for each character. The
anti-DOS measures in rtrek will force a one-second minimum
wait between each call in the event that the individual calls actually
return results faster than this, so the code below will take at least
seven seconds to complete.
Add a fun Star Trek-themed plot? Make it so.
characters <- eaf$characters %>% select(uid, name) %>% filter(name %in% characters)
characters
#> uid name
#> 1 CHMA0000123101 Deanna Troi
#> 2 CHMA0000123073 William T. Riker
#> 3 CHMA0000278225 Miles O'Brien
#> 4 CHMA0000278224 Natasha Yar
#> 5 CHMA0000123141 Worf
#> 6 CHMA0000132570 Geordi La Forge
#> 7 CHMA0000261620 Data
#> 8 CHMA0000289509 Jean-Luc Picard
#> 9 CHMA0000123143 Beverly Crusher
eps_count <- sapply(characters$uid, function(i){
stapi("character", uid = i)$episodes$series |>
summarize(sum(title == "Star Trek: The Next Generation")) |>
unlist()
})
eps_count <- select(characters, name) |> mutate(n = eps_count)
library(ggplot2)
library(showtext)
font_add("StarNext", system.file(paste0("fonts/StarNext.ttf"), package = "trekfont"))
showtext_auto()
uniforms <- c("#5B1414", "#AD722C", "#1A6384")[c(3, 1, 2, 2, 2, 2, 2, 1, 3)]
eb <- element_blank()
ggplot(eps_count, aes(factor(name, levels = name[order(n)]), n)) +
geom_col(fill = uniforms, color = "gray20") + coord_flip() +
theme_minimal(base_size = 22) +
theme(plot.title = element_text(family = "StarNext"), line = eb, axis.text.x = eb) +
scale_x_discrete(expand = c(0, 0)) + scale_y_continuous(expand = c(0, 0)) +
labs(x = NULL, y = NULL, title = "TNG EPISODE APPEARANCES") +
geom_text(aes(label = n), color = "white", size = 8, hjust = 1.5)
This looks as expected. Inspect the structure of the list objects
returned by stapi to become more familiar with what kind of
information is available.
Minor weirdness
Sometimes you may receive errors when trying filter rows for one of
the data frames while certain problematic columns are still selected.
This is likely because the data frame contains a nested data frame, but
one which is not nested in the typical way (e.g.,
tidyr::unnest will also fail to resolve the issue).
In the code immediately above, this occurs with the
series data frame, which is why the episodes
parent data frame is subset using $series before calling
summarize.