The SIU pipeline: fetch, parse, audit, translate

Overview

The Ontario Special Investigations Unit (SIU) publishes director’s reports on police-civilian interactions within its mandate (death, serious injury, sexual assault, or the discharge of a firearm at a person). rmorie has a pipeline for that corpus:

  • morie_fetch_siu(), which writes the panel-reviewed corpus bundled in rmoriedata or, on request, fetches the reports live;
  • a C++ parser for the English and French report templates;
  • a language-aware DRID manifest that ships inside the package (morie_siu_index());
  • a correction system that applies hand-verified values over the parser output (morie_siu_record_correction());
  • audit and LLM tooling (morie_siu_audit_case(), morie_siu_anomaly_check(), morie_siu_llm_extract(), morie_siu_compare()) and French-to-English translation (morie_siu_translate()).

Chunks marked eval = FALSE below need the network or a running LLM server, and are not run when the vignette is built.

Inspecting the shipped manifest

The DRID manifest ships inside the package, so reading it requires no network access.

library(rmorie)

manifest <- morie_siu_index()
dim(manifest)
#> [1] 4749    9
table(manifest$`_language`)
#> 
#>   en   fr 
#> 2532 2217

# canonical_drid points each French drid at its English counterpart.
head(manifest[manifest$`_language` == "fr",
              c("drid", "case_number", "canonical_drid")])
#>    drid case_number canonical_drid
#> 2    47  17-OVI-201             46
#> 4    65  17-TVI-108             48
#> 6    67  17-TCI-209             66
#> 8    69  17-OCI-220             68
#> 10   71  17-TCI-228             70
#> 12   73  17-OCI-243             72

The corpus

morie_fetch_siu() returns the path of an SIU.csv file. By default it writes the panel-reviewed corpus from rmoriedata:

csv_path <- morie_fetch_siu(lang = "en")
siu <- utils::read.csv(csv_path, colClasses = "character")
dim(siu)
#> [1] 2309   66

With options(morie.siu.allow_fetch = TRUE) the function fetches the reports live instead. It walks the manifest, requests each report and news release through a rate-limited pool (rate_rps requests per second, concurrency parallel transfers, up to 3 retries per request), interleaving the report and news-release requests, and parses the pages with the C++ parser:

options(morie.siu.allow_fetch = TRUE)
csv_path <- morie_fetch_siu(
  lang       = "en",   # skip French drids: half the network traffic
  cache_html = TRUE,   # keep the pages locally for audits and re-runs
  rate_rps   = 4,      # requests per second
  overwrite  = TRUE
)

Auditing a single case

morie_siu_audit_case() returns the parser row, the raw report and news HTML, and the cleaned text side by side, so you can see exactly what the parser saw. It fetches the pages if they are not cached:

audit <- morie_siu_audit_case("17-OVI-201")
names(audit)
# "row" "drid" "nrid" "report_html" "news_html" "report_text" "news_text"

morie_siu_anomaly_check() asks an LLM, per field, whether the HTML supports the parsed value, and morie_siu_audit_columns() does the same across a sample of cases. Both need an LLM provider (see below):

morie_siu_anomaly_check("17-OVI-201")
morie_siu_audit_columns(case_numbers = sample(siu$case_number, 50))

Comparing parser output against an external table

morie_siu_compare() produces a side-by-side diff between the parser row and any external table that names the same case. field_map maps external column names to parser field names, and external_case_col names the external table’s case-number column:

my_other_table <- data.frame(
  case_number = "17-OVI-201",
  n_officers  = 1L
)

morie_siu_compare(
  case_number       = "17-OVI-201",
  external          = my_other_table,
  field_map         = c(n_officers = "number_of_subject_officials"),
  external_case_col = "case_number"
)

LLM extraction

morie_siu_llm_extract() runs an LLM over a cached report and returns the parser’s 64 columns as extracted by the model. The default model = c("ollama", "route", "gemini") tries a local Ollama server first, then the model configured with morie_llm_config(), then Gemini (paid, GEMINI_API_KEY or GOOGLE_API_KEY). A local Ollama server needs no API key.

morie_siu_llm_extract("17-OVI-201", model = "ollama")

Ollama settings: OLLAMA_HOST (default http://localhost:11434), OLLAMA_MODEL (otherwise the first model the server lists), and OLLAMA_API_KEY for hosted gateways.

French -> English translation

morie_siu_translate() translates the text fields (narrative_summary, news_release_summary, news_release_title, relevant_legislation) of cached French reports through the LLM provider:

morie_siu_translate(target_lang = "en", case_numbers = "17-OVI-201")

Corrections: the parser learns

Ship-time corrections live in inst/extdata/siu_canonical_overrides.csv.gz:

ov <- utils::read.csv(system.file("extdata", "siu_canonical_overrides.csv.gz",
                                  package = "rmorie"))
nrow(ov)                       # corrections
#> [1] 47
length(unique(ov$case_number)) # cases they cover
#> [1] 12

Users can add their own corrections, which take effect on the next morie_fetch_siu() run:

morie_siu_record_correction(
  case_number    = "17-OVI-201",
  field          = "number_of_officers_involved",
  verified_value = "1",
  note           = "checked against the report"
)

Overrides are applied per cell, by case number, immediately after the parser writes its initial output.

Format validity

morie_siu_sanity_check() validates per-row format constraints: case-number pattern, ISO dates, integer counts, Yes / No values, and narrative columns that are empty or suspiciously short. It expects the columns as character, so read the CSV with colClasses = "character" (as above):

sane <- morie_siu_sanity_check(siu)
sum(sane$issues_count > 0)   # rows with at least one flag
#> [1] 31
head(sort(table(unlist(strsplit(sane$issues[sane$issues_count > 0], ";"))),
          decreasing = TRUE))
#> 
#>                  age_affected:not-int  number_of_civilian_witnesses:not-int 
#>                                    14                                    10 
#>         date_siu_notified_iso:bad-iso narrative_summary:empty-when-expected 
#>                                     3                                     2 
#>                case_number:bad-format  narrative_summary:suspiciously-short 
#>                                     1                                     1

Where to go next

  • vignette("mrm-dataset-fetchers") covers the OTIS, TPS and case-law fetchers.
  • vignette("mrm-empirical-callables") runs mrm_siu_case_to_decision_km() and mrm_siu_per_service_rate() on this corpus.
  • morie_siu_classify_mandela() applies the Mandela-Rules classifier to federal Structured Intervention Unit stays; see the mrm-otis-walkthrough vignette.
  • Citation: see citation("rmorie").