morie_grid_search_cv() use separately
named deviance helpers, which clears the R CMD check NOTE "multiple local function definitions
for dev_fun" reported on every r-universe platform.stat_bridge_main("verify") and stat_bridge_verify(execute = TRUE) run each command on
sample inputs (p-values, x and y, a correlation matrix), as the Python self-test does,
instead of calling it with no arguments, which reported every command that takes input as
FAIL; a command with no sample input is shown as NEEDS INPUT. The bridge tests remove the
commands they register.morie_capsule_write() pins a local CSV in data_provenance.json (file name,
SHA-256, size, data rows, source URL, retrieval date), morie_capsule_verify() checks it,
morie_capsule_sign() and morie_capsule_bundle() sign a digest, a manifest or the whole
directory (ML-DSA, SLH-DSA, XMSS or HMAC), morie_capsule_drift() compares a fresh copy with
the pinned one, and morie_capsule_manifest() and morie_capsule_compare() record results and
recompute them, through rmoriebricklayer; the same layer is in morie and in rmbl-action. New
vignette capsules.morie_capsule_write() writes name, source, resource in that order, as the rmbl-action
examples and morie do, names the capsule after the data file (data.csv -> data) by default,
and takes table and script_file as morie.capsule.write() does. morie_capsule_verify()
reports a provenance record that does not parse as a failed provenance_readable row, as morie
does; rmoriebricklayer::vrfcps() stopped with an error.morie_llm_login() passes accept_terms, affiliation, purpose, restricted and the
supervisor_* arguments to rmoriebricklayer 1.0.0 under its names (accter, affil, ...);
they were refused as unused arguments.morie_download() returns the path, and errors when the live file and the Wayback Machine
both fail (it returned TRUE or FALSE, and FALSE silently). attempt_wayback = TRUE looks
a snapshot up; it switched the fallback off, so the CKAN file route never reached the archive.morie_emissions_verify() documents that any other file in the directory fails the
no_unlisted_files check, and morie_emissions_start() points to rmoriebricklayer::pqckg()
(there is no pqc_keygen()).rmoriebricklayer::siuprs(), rmoriedata::mdload() and so on).research/lean/IndigenousWellness: 120 Lean 4 theorems (Mathlib, no unproved step) for the
properties the First Nations, Metis and Inuit wellness estimators in rmoriebricklayer rely on.rmorie login and
morie_llm_login() (through rmoriebricklayer 0.5.14) show the terms, ask Do you accept ...? [y/N],
and then ask for your name, organisation and why you need the data; the restricted tables also
need a supervisor, department chair, director or coordinator. Give them as flags
(--accept-terms, --name, --reason, ...) or as arguments of morie_llm_login().Live progress in the long-running work, through rmoriebricklayer 0.5.13's
bricklayer_progress(): the bootstraps (morie_boot(), morie_boot_run(), morie_tsboot(),
morie_bootstrap_sample()), the honest causal forest and its bootstrap, the random forests,
the DML cross-fitting (morie_otis_irm_dml(), morie_dml_clustered(),
morie_caus_dml_partial_lin()), super_learner(), morie_otis_aipw_superlearner() and the
ten-estimator OTIS run. The bar shows only when someone is watching (an interactive session,
or options(morie.progress = TRUE)), after the work has run for a moment; nested work shows
one bar, and options(morie.quiet = TRUE) silences it.
The vignettes were rewritten against the code: every chunk runs, none prints NULL, every
count is recomputed, and the references to an "MRM framework paper", a "companion paper" and a
"package paper", none of which exists, are replaced by the rmorie functions that do the work
(citation("rmorie") for the software). Wrong field names (aipw$estimate and others), the IPW
weights of ipw-deep-dive, and claims about functions (morie_estimate_ate() is one IPW
estimate, not an ensemble) are corrected.
morie_otis_analyze_a01_mrm() and the other Ruhela formulations say when they run on synthetic
rows (the 5-row sample morie_otis_load() falls back to, or morie_synth_otis()) or on a
treatment with an empty arm, and then give no interpretation; their printed results show every
estimate table, and no result text claims a finding the data did not produce.
The live OTIS loaders read CKAN's datastore/dump with its column names (they came back as
V1..V4 with the row id); the b05 and c11 samples use the real duration bins, and a table whose
bins are unknown is reported, not summarised as nothing.
Faster on the full OTIS a01 (65,467 person-years): g-computation's bootstrap 55 s to 2 s, the SuperLearner's forests 1,218 s to 314 s (exact node sort; rows that share a covariate pattern grow one weighted row), with the same estimates.
morie_lmm() no longer stops on the boundary when nlminb reports singular convergence (Orthodont
with a 1e-10 perturbation of y missed the REML maximum in 31 of 60 draws); native gbm
propensities break an exact tie in split gain the same way on every platform.
Fixed, each with a test: mrm_siu_outcome_classifier() on the shipped corpus;
morie_siu_sanity_check() on a default read.csv(); TPS spatial measures drop the (0,0)
placeholder and report how many; the Doob table summaries are computed; morie_e_value()
returns e_value; morie_dag_estimate() has unique names; morie_dif_delta_mh() labels
favours the right way round; morie_run_propensity_ipw_analysis() applies the survey weight;
q75_days is named for what it is; morie_otis_irr_glmm_vm() uses the a01 definitions;
morie_siu_compare() names unmatched fields.
Needs rmoriebricklayer 0.5.13 (its live progress bars and the language-model layer this release is built on) and rmoriedata 0.3.6; CI installs rmoriebricklayer from its v0.5.13 tag, not a branch.
Language models are not hosted any more: bring your own. morie_llm_ask(),
morie_llm_ask_multi(), rmorie ask, rmorie models, rmorie config and rmorie doctor
use the route rmoriebricklayer (>= 0.5.13) sets up: your own OpenAI-compatible endpoint, each
provider whose key is set (OpenAI, Anthropic, Google Gemini, OpenRouter, Groq, Mistral and
more), or a local Ollama. The model is given the help of rmorie, rmoriedata and
rmoriebricklayer, with tools to search and read it. morie_llm_hosted_models() and
morie_llm_probe_hosted() are gone, and route = "hosted" is an error that says so. With no
model set up, the answer is the keyword fallback, marked as such.
morie_siu_llm_extract() and the other SIU model calls take "route" (or a provider id such
as "anthropic") for the model you set up; the default chain is c("ollama", "route", "gemini").
rmorie login is for the curated tables at data.rmorie.com only (GitHub or an emailed code
issues the key). A downloaded table is checked against the SHA-256 the manifest lists, and a
restricted table (ICPSR, NCVS 38604, a2aj) answers with where to accept its terms.
Research ledger, the third and last library-scan batch (20 Lean files, 84 audited theorems; 517
theorems over 103 files, 0 sorry) with R functions and tests in R/ledger_scan2.R:
morie_nb_matches_zip(), morie_gamma_bound(), morie_blocking_crossover(),
morie_adjustment_variances(), morie_budget_optimum(), morie_outcome_test(),
morie_spatial_omission(), morie_iv_natural_bounds(), morie_design_sensitivity(),
morie_moran_trend(), morie_exposure_positivity(), morie_permutation_valid(),
morie_thinning_identification(), morie_poisson_chernoff(), morie_random_walk_exposure(),
morie_extremal_index_clusters(), morie_tail_ratio(), morie_calibration_flip(),
morie_bayes_risk_dpi(), morie_bellman_value(), morie_myopic_example(),
morie_budget_rule() and morie_run_length_bound(). Every item of the 2026-09-21 library scan
now has a machine-checked core.
Research ledger, the second library-scan batch (12 Lean files, 57 audited theorems; 434
theorems over 83 files, 0 sorry) with R functions and tests in R/ledger_scan.R:
morie_attenuation_two_regressor() (A4), morie_friedman_urn() (C2),
morie_ruin_probability() (C4), morie_cox_overdispersion() and morie_zip_vuong_boundary()
(C12, B24), morie_nonresponse_region() (B10), morie_instrumental_inequality() (B12),
morie_misclassified_rd() and morie_misclassified_contrast() (B14),
morie_sensitivity_amplification() (B18), morie_design_effect_direction() (A10, B9),
morie_ht_elephant() (B8), morie_attractiveness_contraction() (C7) and
morie_inspection_game() (A18, C24). Each function carries the theorem it computes from as
its theorem attribute.
Victimisation data and the applications that waited for it: the NCVS 1992-2022 concatenated
file (ICPSR 38604; incident, person and household files, the person and household files in two
year ranges each) is served at data.rmorie.com as ncvs_38604/* with the ICPSR codebooks, user
guide, citation and terms of use beside it (rmorie pull ncvs_38604/incident).
research/applications/ncvs_victimisation.R runs the ledger on it: the P1 survey/record pair
(victimisations against the ones the police learned of, by year and class, with the
partial-identification interval and the breakdown value of the 1993 to 2022 decline), the P7
concentration of victimisations on persons and households against the Poisson null, the P10
moments contagion and heterogeneity fit alike, and the A14 comparable-subset rate.
Research ledger, fifteen more items closed with machine-checked proofs (research/lean, now 71
files, 377 audited theorems, 0 sorry): the recording rate is identified only on the
survey-comparable offence set, with the dark-figure interval for the rest
(morie_dark_figure_comparable(), library scan A14); the stochastic urn's share converges
almost surely and in mean, each step being the conditional expectation of the limit, so the
long-run allocation is a random variable with the starting share as its mean (Lean
Research.P4.Martingale, on Mathlib's martingale convergence theorem); the hot-spot threshold of the Short-Brantingham
reaction-diffusion model, the unique steady state and the dispersion relation with an explicit
stability inequality (morie_hotspot_threshold()); the ring mis-specification bound for the spillover
estimand (morie_spillover_ring_bound()); the hypergeometric rank-persistence null
(morie_rank_persistence_null()); the stochastic urn's share has variance at most
a/(4(r0+b0)) at every horizon, so the loop drifts within a Chebyshev band fixed by the starting
mass (morie_feedback_loop_envelope()); Albert-Anderson's converse, overlap gives a finite
logistic maximum-likelihood estimate (morie_logit_separation()$mle_exists);
four ledger problems applied to public data (research/applications/: Toronto neighbourhood
concentration against the Poisson null, break-and-enter over-dispersion and per-neighbourhood
Hawkes fits, the OTIS gender gap in placement lengths split by DFL and RIF, the neighbourhood
contiguity graph as the interference design), the rest listed as open with the reason;
morie_weightit() for a continuous treatment follows WeightIt 2.x: the marginal density in the
numerator is the sampling-weighted mixture of the conditional densities on a grid of the
treatment, splined at the observed values (the 1.x normal marginal gave weights 7 to 10 percent
larger); ten help pages carried "See Usage" placeholders beside the documented argument;
morie_logit_separation() now settles complete versus quasi-complete separation by a second
programme that maximises the smallest margin, so a vertex of the first programme sitting on a
zero margin no longer reports completely separated data as quasi-complete; about 25 help pages
had two @param tags on one line, folding the second parameter into the first description; the veil-of-darkness odds ratio is the relative
visible-race factor and a sign test (morie_veil_of_darkness()); the three-list never-listed
cell under no three-way interaction and its exp(-t) sensitivity
(morie_dark_figure_no_interaction()); the error of the plug-in shrinkage factor
(morie_shrinkage_factor_error()); the Gini is at most the coefficient of variation over
sqrt(2), so a Poisson null caps it at about 1/sqrt(2 mean) (morie_concentration_null_bound());
the expected-cost deterrence response is linear in the log design with the discount as a
coefficient ratio, and the Wald ratio identifies one dimension under exclusion and relevance
(morie_deterrence_logcost(), morie_wald_iv()); the quantile RIF averages to the quantile
exactly at the sample quantile and carries the gap (morie_rif_quantile(), morie_rif_gap());
two count laws with equal mean and variance but different zero mass, the second-moment core of
Bartlett's non-identification (morie_contagion_moment_witness()); the cross-age covariance of
a type mixture that the aggregate age-crime curve cannot see (morie_age_crime_type_cov(),
morie_age_crime_panel_cov()).
morie_weightit(): balancing weights under WeightIt's method names with no call into it,
for binary, multi-category and continuous treatments: "glm" (any binomial link; maximum
likelihood, Firth, ridge, lasso, elastic net; multinomial or cumulative-link; the
density-ratio weight for a continuous treatment), "ebal", "ipt" and "cbps" (just- or
over-identified), every get_w_from_ps estimand, sampling weights, stabilisation, balance
moments and interactions. morie_causal_weighting() now runs "ebal" for every estimand
and "ipt" natively. The engines had been in the package without an entry point; each is
tested against the equation it solves and, where WeightIt is installed, against WeightIt.
SIU: morie_siu_llm_extract() and morie_siu_anomaly_check() on a case with no harvester
cache (the self-serve route) now carry the resolved drid and the cache row's nrid instead of
failing on the bookkeeping columns; a provider that answers with an empty candidates, content
or choices array is reported as "returned empty text" rather than a subscript error; an empty
verdict array from the auditing model is an empty audit table.
OTIS: morie_otis_analyze_a01_ruhela_subgroup_male() and the b01 sibling refuse a subgroup
under 100 cells with the same message as the female formulations.
Dataset fetches: an HTTP error whose JSON body carries the message as the error string
itself is reported with that message.
morie_siu_reports(update = TRUE): a field name the parser does not emit (the pre-0.5.5
number_of_subject_officers) read as absent instead of a subscript error that abandoned
every newer report, so the refresh now appends them.
Tests cover the TPS hub wrappers without mocking, the advanced Hawkes drivers and their
base-R fallbacks against the C++ paths, the statistical-physics routines on exact synthetic
frames, morie_fetch_siu() and the SIU review side with the transport replaced by recorders,
the dataset HTTP helpers on both backends, and the OTIS Mandela, chi-square, grid and master
analyses with every reported number recomputed.
Research ledger, three open items closed with machine-checked proofs (research/lean, 273
theorems and lemmas, 0 sorry): one recorded crime series cannot separate re-classification
from cuffing and a second source's true total can (morie_recording_regimes()); the
Imbens-Manski cutoff exists and is unique (morie_bounds_confidence() cites
im_cutoff_exists, im_cutoff_unique); across judges nested by leniency a release rule's
identified interval narrows and collapses exactly on containment
(morie_selective_labels_judges()). The Lean library directory now holds all 56 proof files.
Every estimator now runs on the package's own code; the reference packages are used only by
tests that check the numbers (the opt-in backend = of morie_taphonomy_bhm() and
morie_causal_weighting()'s methods beyond glm/ps/cbps/ebal are the two exceptions you ask
for by name). Newly native, each with tests against the package it replaces:
morie_roc_auc_score() (pROC's thresholds and AUC), morie_decision_tree_split() (rpart's
CART, surrogates included), anova_table() Type II/III (car::Anova), morie_grid_search_cv()
and morie_random_search_cv() (caret's folds, draws and scores, with native lm, glm, ridge
and glmnet learners), morie_effects_emmeans(), _predictions(), _comparisons(),
_slopes() and _tidy() (emmeans, marginaleffects and broom for lm and glm),
morie_causal_impact() (a Bayesian structural time series by Gibbs sampling, with
CausalImpact's priors and summary), morie_eg_coint(), morie_johansen_cointegration() and
morie_ts_stationarity() (urca's ADF, Johansen and a new KPSS), morie_regime_switching()
(MSwM's EM), storey_q(), estimate_pi0() and the p-value combiners (qvalue, poolr),
jackknife() and morie_rsample_bootstraps(), the superlearners' random forest, the
Aldrich-McKelvey and blackbox scalings (basicspace), morie_logit_separation()'s LP
(lpSolve), morie_matching_estimate_propensity(model = "gbm") (gbm with bag.fraction = 1),
DBSCAN in the TPS clustering and rendering functions (dbscan, including points exactly eps
apart), gam_smoother() (mgcv), and the pseudoinverse in the geron learners (MASS).
Before, several of these gave different results depending on which packages were installed,
and some stopped when one was missing. The fairness GAN functions no longer require torch,
which they never used.
Changed results and shapes: morie_rsample_bootstraps() returns a morie_bootstraps data
frame; morie_effects_emmeans() a data frame shaped like emmeans' summary;
morie_regime_switching() returns the log-likelihood (not its negative) and n smoothed
probabilities; storey_q() estimates pi0 over a lambda grid by default, as qvalue does;
n_effective_tests() rounds down as poolr does; the CV searches report the chosen
candidate's score (the regression score was the worst one before).
DBSCAN's neighbour search uses a grid of eps-sized cells in one to three dimensions: 30,000 points cluster in about a second instead of half a minute.
New native estimators, written in R with no outside package: morie_lmm() (linear mixed
models as lme4::lmer, with nlme::lme's random = form and containment degrees of
freedom), morie_glmm() (binomial and Poisson GLMMs by the Laplace approximation, as
lme4::glmer), morie_gam() (thin plate and cubic regression splines with GCV, UBRE or
REML smoothness selection, as mgcv::gam), morie_rq() (quantile regression by
Frisch-Newton interior point or Barrodale-Roberts, with nid, iid, kernel and bootstrap
standard errors, as quantreg::rq) and morie_nls() (Gauss-Newton and Levenberg-Marquardt,
as stats::nls). Each has print, summary, coef, vcov, fitted, residuals and
logLik methods, and tests that check it against the reference package when that is
installed. Mixed models factor one small block per group level (a Schur complement
handles a second grouping factor) instead of a dense n x n matrix, so a model that took
14 s with the previous dense code takes about 1 s; a two-smooth GAM on 10,000 rows takes
under a second.
Install routes: Remotes: no longer pins [email protected] and [email protected],
so remotes::install_github("rootcoder007/rmorie") builds the current companions
instead of those old tags, and the floors are raised to rmoriebricklayer (>= 0.5.12) and
rmoriedata (>= 0.3.6). The README install snippets name the companions and pass repos
to every install.packages() call (under Rscript there is no mirror chooser, and a bare
call stops with "trying to use CRAN without setting a mirror"), and the GitHub route
upgrades "always" so an older companion already installed is replaced.
The README's R install routes lead with pak (download progress, compiler output hidden
unless a build fails, named packages upgraded to the current release), with
install.packages() and remotes as the fallback; a new install-routes workflow runs every
README one-liner on Linux, macOS and Windows.
Language models: a running Ollama server with no model pulled no longer counts as a
usable route. morie_llm_detect_provider() picked it (the probe only asked whether
/api/tags answered), so rmorie doctor reported "active provider: ollama" and ask
first sent a request with no model name; the automatic route now skips such a server
(unless OLLAMA_MODEL names a model) and goes on to the next route. New morie_llm_config() and rmorie config (show, help, get, set,
unset, setup, path) save the route (auto, own, ollama or a provider id) and the address,
key and model of each route in ~/.config/morie/llm.json (private, written only on request;
the same file and keys as rmoriebricklayer's rmbl config); an environment variable that is
set still wins. A route other than auto is the only one asked, and says what is missing
when it is not set up. rmorie ask --route ROUTE picks one for a single call, rmorie doctor ends with the route and model ask will use, and rmorie help start | llm | config | r are step-by-step guides. The Ollama route now sends OLLAMA_API_KEY (the probe and
the chat request did not), and an endpoint given as .../v1 (as rmorie provider set
documents) is no longer asked at .../v1/v1/chat/completions; Gemini's endpoint had the
same doubled path.
CI: the daily r-cache-warmer pre-installs rmoriebricklayer and rmoriedata from r-universe, like the other jobs. pak runs vanilla R and only sees the CRAN mirror, where the companions are older than the floor, so the warmer had failed since 1.4.1.
AdvectionDiffusion2d() documented a stability criterion that was false (cfl
below 1 and diffusion number below 1); the explicit upwind-plus-FTCS scheme needs
|u dt/dx| + |v dt/dy| + 2 (kx dt/dx^2 + ky dt/dy^2) <= 1, and a run outside it reached
10^10 from a unit pulse while reporting itself stable. The bound is returned as
stability_number and stable; a call outside it is taken in enough sub-steps of a
smaller dt to satisfy it (reported as substeps and dt, with a warning), so the field
is a solution over the same physical time rather than an explosion.n_images a
whole number (2.5 was truncated silently), the stack hotter than the air for Briggs' rise
(a cooler stack returned NaN), x0 and y0 single numbers (a vector doubled mean_x),
a warning when the release is above the mixing height. GaussianPlume(q = -100) returned a
negative concentration, u = 0 returned Inf, GaussianPuff(t = -100) a negative one and
PgSigmas(-500) a negative sigma.MORIE_COUNTRY_ISO and the geolocation
lookup, and whenever the lookup is skipped (MORIE_EMISSIONS_OFFLINE) or fails, the
system time zone is mapped to its country through the IANA zone tables (every zone of
zone.tab plus every alias in backward, 549 names) and then the locale's territory,
before falling back to the world average. Two-letter codes are mapped to the energy
table's three-letter codes through the full ISO 3166 list instead of a hand-picked map of
thirty countries.pull --out DIR, a directory: says "cannot write DIR: it is a directory" without R's
"not a regular file" warning ahead of it; a file it cannot open is reported once, by
the error that names the cause.Sign-in and errors, after rmoriebricklayer 0.5.9's network review: the page the sign-in service names is handed to the browser only when it is an https address of a public host, and a gateway's error text is redacted of the key it was sent (by value and by shape) before it becomes an R condition.
The hosted MORIE tier is the last resort: morie_llm_detect_provider() and the fallback
chain in morie_llm_ask() / morie_llm_ask_multi() now try a local Ollama, then your own
Gemini / OpenAI-compatible / OpenAI keys, and only then the hosted tier. Its address, model
and sign-in service come from the signed services document at rmorie.com (through
rmoriebricklayer::bricklayer_services(); an older bricklayer keeps the previous defaults),
so they can change without a release. Keys are personal and issued on request at
https://rmorie.com/access; every hint says so (rmorie login --token, GitHub and emailed-code
sign-ins keep working). The curated-data address follows the same document.
The 19 OTIS Ruhela-formulation analyses that returned a "not yet ported" placeholder run:
morie_otis_analyze_{a01,b01,b02}_ruhela_formulations() (the ten-estimator DLRM with the IRM-DML
standard-error comparison and the Naive arm), the per-year driver and its a01/b01 forms, the a01,
b01 and b02 alternative-treatment and gender-subgroup formulations, and
morie_otis_analyze_a01_with_csi_context() (TPS tables passed as tps_data= or loaded). New
estimators behind them: morie_otis_gcomputation(), morie_otis_atc(), morie_otis_balance()
and morie_otis_per_year_irm_dml(). An aggregate analysis given a table without its columns says
which columns are missing.
Weak-instrument tests computed from the data: morie_iv_montiel_olea_pflueger() (effective F,
robust to heteroskedasticity, clustering and serial correlation, with Patnaik critical values for
the simplified, TSLS and LIML tests) and a native morie_iv_kleibergen_paap() (rk statistic and rk
Wald F by the Kleibergen-Paap SVD, robust or clustered; it returned the Cragg-Donald statistic).
morie_iv_2sls() reports the effective F and its critical value beside the iid quantities, and
morie_iv_stock_yogo() outside its rows points to both instead of "TODO: extend".
The spatial-voting estimators that returned a substitute run their own models, each checked
against its reference implementation: morie_spatial_voting_alpha_nominate() (Carroll et al.
2013 slice sampler; it returned an EM-IRT fit; agrees with anominate on alpha, beta and the ideal
points, r = 0.9998), morie_spatial_voting_bayesian_am() (Hare et al. 2015; it returned
basicspace's deterministic AM; matches the authors' JAGS model to the third decimal),
morie_spatial_voting_bayesian_mds() and morie_spatial_voting_bayesian_unfolding() (Bakker and
Poole 2013 lognormal model; they returned smacof's stress solution), morie_spatial_voting_ordinal_irt()
(Quinn 2004 with item cutpoints; it reported MCMCpack's loadings as ideal points, and its fallback
shared one set of cutpoints across items and rescaled the draws every sweep) and
morie_spatial_voting_dynamic_irt() (Martin and Quinn 2002 Gibbs sampler). Without basicspace,
morie_spatial_voting_aldrich_mckelvey() computes Aldrich and McKelvey's closed form (it fitted a
different regression) and morie_spatial_voting_blackbox() fits the observed cells only (it filled
missing cells with zeros); both now match basicspace, which morie_spatial_voting_blackbox()
never reached because it passed missing = NA.
Every exported function has a runnable example, and a bad first argument (wrong type, shape or missing values) stops with a plain-language message naming the argument.
Without DBI, a cache given as db_path= keeps its files beside that path; every such cache shared
one directory, so a table cached for one database was read back for another.
morie_estimate_ate() (and the MRM "ipw ate" row) reports the HC3 sandwich standard error of the
weighted regression y ~ t whose coefficient its Hajek estimate is, as morie's Python does. It
used the Horvitz-Thompson influence function, which is not centred on the group means: the
standard error ran 45-57% above Python's and 69% above the sampling SD in simulation (95%
intervals that covered the truth 100% of the time). The two arms now agree to 1e-12.
morie_estimate_double_ml() cross-fits both nuisances on one sample split, its folds drawn from
the splitmix64 uniforms rmoriebricklayer and morie's Python share (R's sample() and numpy's
permutation never agreed, so one seed gave different folds in the two languages).
morie_kendall_tau() and morie_wilcoxon_signed_rank_test() use base R's default p-values (exact
below 50 pairs without ties), as morie's Python does; they forced the normal approximation.
exact = chooses.
morie_hawkes_fit() fits 50 events or more with rmoriebricklayer::core_hawkes_fit() (bounded,
analytic gradient): 1,083 Weibull or Lomax events take under a second instead of 32-38 s, and a
Lomax fit that tends to its exponential limit stops at the bound with a note that its shape is not
identified, instead of reporting alpha = 4.8e8. The result also carries at_bound.
morie_tps_hawkes_advanced_fit() refuses a kernel/method pair no fit can serve (soe on a kernel
that is not completely monotone, inar with a sinusoidal baseline) with an error, as Python does;
it returned an empty result with the reason in $warnings.
rmorie analyze tps runs: '{"datasets":["Assault"],"nrows":5000}' (names as morie_tps_load()
takes) or '{"data":"FILE.csv"}'; without either it exits 1 with the usage (it exited 0 with
"not_available").
morie_datasets_ckan_package() read no package (the resource list arrives as a data frame) and
now returns every CSV, a bilingual package's English and French copies under "(en)" and "(fr)";
CKAN CSVs in Windows-1252 (Ontario's library statistics) are read instead of stopping on "invalid
multibyte string".
A read-only or full home directory no longer stops pull --out or a loader: the cache is skipped
with a message naming the directory, and the data are returned; a failed cache write names the path
instead of a later "cannot open the connection".
pull cihi820b and the other CIHI workbooks: a jurisdiction merged down its block is carried to
every row (112 of 133 rows were blank), Excel's number noise (51.959413779999998) is written as
the workbook's digits, and pull writes CSV as morie's Python does (quotes only where needed, an
empty field for a missing value), so the two arms' files compare equal. The progress line names the
dataset, one line each.
download-bootstrap keeps a table that is already cached (--refresh downloads again), keeps one
copy instead of two 230 MB files, and says "the CKAN datastore answered HTTP 500" rather than
"unreachable" when the server answered.
spwkth() computes its integrated density in milliseconds (the omega integral is exact, not a
20,001-point quadrature: 37 s before). Slow examples were trimmed and the morie_entheo_* examples
say how to get the DMT_Imaging data instead of printing an error.
DESCRIPTION requires rmoriedata (>= 0.3.4) and pins Remotes to [email protected] and [email protected]; the README installs both from r-universe (CRAN carries older versions) and describes the 0.3.4 SIU corpus (4,613 reports, 66 columns, English and French).
Security: morie_crypto_hybrid_encrypt() and rmorie crypto encrypt now keep files private.
The R arm still wrote the 1.3.x container, whose wrapping key was derived from the KEM ciphertext
and the public key alone (the shared secret was computed and discarded), so anyone holding the
file and the PUBLIC key could open it. The wrapping key now comes from
HKDF(shared_secret || kem_ct || pk) behind the 1.4.0 marker MORIEHYB 0x02, the same bytes
morie writes: rmorie and morie open each other's files, and a 1.3.x file still opens, with a
warning to encrypt it again. Files encrypted with rmorie 1.3.x or an earlier 1.4.0 build should
be encrypted again. morie_crypto_hybrid_container_version() tells the two apart.
rmorie crypto keygen --output DIR writes marked key files (MORIEPK/MORIESK, as morie) and
the secret key owner-only (mode 600; it was world-readable), and refuses to replace an existing
secret key without --force. decrypt --key takes a .moriesk file as well as a keystore name
(a key pair from keygen --output could not decrypt anything); encrypt and decrypt take
--out and refuse to overwrite an existing file without --force; the wrong kind of key file,
a missing one and a 1.3.x public key are each refused with their own message.
morie_mrm_estimate_causal_effect()'s consensus standard error is the weighted mean of the
estimators' standard errors. It pooled them as independent studies (sqrt(1 / sum(w))), but all
four run on the same rows, so the interval was about half as wide as it should be: in 300
simulated data sets (n = 500, true effect 0.8) its 95% interval covered the truth 70% of the time
(morie's run); it now covers 96%.
morie_matching_att_matched() counts a control matched to several treated units once per pair
squared (Abadie and Imbens 2006), not once per pair: with matching with replacement its interval
covered the truth 83% of the time in the same simulation; it now covers 93% (95% in morie). With
every control used once the standard error is unchanged.
Citations without a title now carry the publisher's title, and the citation gate also catches a reference written "Author (year), \emph{Journal} ...", which it had missed.
morie_mrm_estimate_causal_effect()'s matching estimate matches with replacement and takes the
ATT from the matched pairs. Without replacement, when controls were no more numerous than treated
units, nearly every control was used and nothing was balanced: on the package's own simulated
design (true effect 0.8) it returned the unadjusted difference, 1.13, while IPW, AIPW and DML gave
0.85. It now gives 0.92, the same ATT and standard error as morie to ten decimals.
SIU police_service is the service of the subject officials, read from the director's analysis
(rmorie's own copy of the SIU core, as in rmoriebricklayer 0.5.5), not the force that notified the
SIU.
Without jsonlite, JSON is read by rmoriebricklayer's port of jsonlite's reader, so every caller
sees the same object either way: the Socrata by-id loaders returned list() on the default
install, because rows carrying a nested location object were left a bare list of records.
HQC (morie_crypto_hqc_keygen() / _encaps() / _decaps()) runs without liboqs through
rmoriebricklayer's native HQC-1 (v5.0.0, a 32-byte secret; liboqs builds use round 4, 64 bytes);
the ML-KEM and ML-DSA wrappers check key and ciphertext sizes before either backend, with the same
words. The post-quantum and hybrid tests run on every install, and the network-gated tests probe a
web host on port 443 (port 53 answered only on DNS servers, so 13 of them could never run).
morie_granger_test() and morie_transfer_entropy_gaussian() refuse a constant or perfectly
predictable response (an exact fit leaves round-off, not 0, in the residual sum, so the test
passed and returned noise), with the same values as morie's Python on every other series.
Hawkes fits in seconds instead of tens of minutes. morie_tps_hawkes_advanced_fit(),
morie_tps_compare_hawkes_kernels() and morie_tps_hawkes_markovian_vs_nonmarkovian() fit
through rmoriebricklayer's core_hawkes_fit() (analytic gradient, projected BFGS in C++, the
routine morie's Python calls) and take a method: "exact" (Ozaki's O(n) recursion for the
exponential kernel), "soe" (Lomax, and gamma with shape < 1, as sums of exponentials with
relative error eps; Beylkin & Monzon 2010), "truncate" (kernel tail mass below eps left
out), "em" (Veen & Schoenberg 2008) and "inar" (Kirchner 2017); "auto" is exact for the
exponential kernel, truncate for Weibull and soe for Lomax and gamma. The reported nll is always
the exact likelihood, so AIC compares across routes. They fit every event by default (max_n = NULL; a requested subsample is deterministic), prepare the event times as morie does, and the KS
p-value is exact up to n = 10,000, so the two arms return the same estimate on the same data.
Native engines everywhere a result could depend on what is installed. Survey designs are native
(morie_survey_design() takes strata, clusters, nest and a finite-population correction; the
mean and GLM standard errors are the Taylor linearisation of survey::svyrecvar, equal to
survey to 1e-12; a survey::svydesign object is read into the same form). Before, an
installed survey changed the object type and stratified or clustered designs fell back to
unclustered SEs. morie_matching_full() is exact optimal full matching (a minimum-weight edge
cover; its total equals optmatch's at a tight tolerance and is below it at the default one),
morie_matching_subclassify() and morie_matching_variable_ratio() follow MatchIt's rules with
identical subclasses, pairs and weights, and the OTIS match_first step uses the same matcher
whether or not MatchIt is installed (it used a random-order logit match without it).
morie_estimate_g_computation() and estimate_ate_gcomputation() report stdReg's sandwich SE,
computed natively (equal to stdReg::stdGlm to 1e-16); before, without stdReg the SE was
sd(mu1 - mu0) / sqrt(n) or a bootstrap, which leaves out the outcome model's uncertainty.
morie_causal_weighting() is native for "glm", "cbps" and ATT entropy balancing and uses
WeightIt only for other methods. Changed results: morie_matching_nearest_neighbor() with
n_neighbors > 1 now matches in rounds as MatchIt does (every treated unit gets its first
control before any gets a second), pair-identical to MatchIt; it gave each treated unit all its
controls in turn before. Bad input: BayesOutbreak(), DlaAggregate(), morie_dsp_ruler_fd()
and two prime helpers no longer hang, DiffEnt() no longer opens a PDF device, and
morie_siu_refresh_manifest() checks out_path before its 6,000-request crawl. All six CIHI
workbooks read without readxl (only a binary .xls needs it), with the real header row even
when a note widens the sheet. Two-proportion effect sizes add the risk difference and odds
ratio beside Cohen's h, and the frequentist test table adds Bonferroni and Benjamini-Hochberg
p-values over the family.
The key store no longer needs the sodium package and its files are now morie's: scrypt and
ChaCha20-Poly1305 are native (RFC 7914 and RFC 8439 vectors), so a store written by the Python
arm opens here and the other way round; entries sealed by rmorie 1.3.x still open. Without
libsodium, ChaCha20-Poly1305, HKDF and the random bytes run on the same native code (identical
output), so the hybrid ML-KEM envelope and rmorie selftest work on every build.
Fresh-user test, fourth pass. morie_psymet_omega() now factors by principal axes and
takes hier from the Schmid-Leiman transformation (promax-rotated factors, one general
factor from their correlations), so a multi-factor scale no longer reports omega
hierarchical near 1. Items that load negatively on the general factor are reverse-keyed and
scored the other way round first, as psych::omega() does (a message names them); on the
psych::bfi items and psych::sim.hierarchical() data it agrees with
psych::omega(fm = "pa", rotate = "Promax") within 0.01 (the rotations differ in detail).
The two-proportion power table applies Kish's design effect of the weights (n_eq_eff,
power_deff) instead of placeholders, and power_srs uses the observed group sizes.
rmorie verify reports a declared placeholder table and a header-only table as such;
rmorie analyze exits 1 when every analysis of the subject failed; a small stratified total
names the strata it leaves empty; ask --model names the model no provider answered for, and
the SIU extraction chain ends with the same cause line; connection warnings no longer leak
before a download error; login says it is still waiting; run-module prints no completion
line when nothing was written; the OTIS grid names the columns a pair lacks; both spellings
of a NAPS key resolve; emissions --country with an unknown code says the location is
detected; morie_siu_resolve_so() accepts a case number. The REML heterogeneity estimate is
the converged optimum (metafor's default stopping threshold leaves it 0.3% off). The native SIU
parser reads ordinal dates ("August 3rd, 2017") and French months, and names the police service from
the sentence that notified the SIU (17-OVI-201 now reads Guelph Police Service, 2017-08-03); SIU page
text decodes in a C locale; ebac-core and ebac-integrations no longer print the expected design-weight
glm warning (other glm warnings still show); the causal-effects table drops an always-empty column.
Research: four new problems join the Lean-backed programme, each with its R function and
tests. morie_meta_random_effects() and morie_meta_dl_bias() (pooling evaluations:
the DerSimonian-Laird truncation is biased upward under homogeneity and the random-effects
variance is never below the fixed-effect one, Research.P13); morie_logit_separation()
(complete or quasi-complete separation makes the logistic likelihood climb without a
maximiser, the Baldus proportionality-review logit, Research.P5);
morie_bounds_confidence() (Imbens-Manski intervals for partially identified sentencing
effects, Research.P11); morie_cheeger_bound() (the conductance of a hot-spot set
bounds the spectral gap of the street graph, Research.P3). The Python package carries
every research function at parity.
Research: seven more problems join the Lean-backed programme, each with its R
function and tests: Duncan-Davis bounds (morie_ecological_bounds(),
Research.P12), monotone treatment selection (morie_sentence_effect_mts(),
Research.P11), the extinction probability of a near-repeat chain
(morie_contagion_extinction(), Research.P10), judge-leniency designs and what the
Wald ratio identifies (morie_judge_iv(), morie_judge_iv_population(),
Research.P14), the Oaxaca-Blinder decomposition of a sentencing gap with both
references and the interaction (morie_disparity_decomposition(), Research.P15),
Little's law on a court docket (morie_court_backlog(), Research.P16) and the
incapacitation identity with its marginal year (morie_incapacitation(),
Research.P17), and selective labels for release rules (morie_selective_labels(),
Research.P18), and regression to the mean at selected hot spots
(morie_regression_to_mean(), Research.P19).
Research: seven continuations close the ledger's open items, each with a Lean theorem,
an R function and tests, at parity with Python: morie_two_point_bound() (Le Cam's
two-point lower bound for the dark figure, Research.P1LeCam), morie_meta_hksj()
(the Hartung-Knapp-Sidik-Jonkman interval with DerSimonian-Laird or REML heterogeneity,
Research.P13HKSJ), morie_judge_slope_test() (the many-judge slope test of
monotonicity, Research.P14Slope), morie_dfl_reweight() (DiNardo-Fortin-Lemieux
reweighting, Research.P15Reweight), morie_backlog_censoring() (the disposed-cases
mean as a bound, Research.P16Censoring), morie_incapacitation_career() (desistance and
replacement, Research.P17Replacement) and morie_hotspot_shrinkage() with
morie_shrinkage_loss() (empirical-Bayes shrinkage and the size of the
regression-to-the-mean fall, Research.P19Shrinkage). The Lean audit now covers 264
theorems in 58 files, 0 sorry, standard axioms only.
The pollution concentration-response coefficients match their sources (both arms): NO2 all-cause
mortality RR 1.02 per 10 micrograms per cubic metre (Huangfu and Atkinson 2020, the WHO 2021 review)
instead of 1.04, and PM2.5 all-cause mortality log-linear at RR 1.08 per 10 (Chen and Hoek 2020,
WHO 2021) instead of an IER triple with no source; the Burnett IER stays for IHD and stroke, so
verify-pollution burdens are smaller than 1.3.x reported.
Fixes from the 1.4.0 fresh-user test agents (tests in test-agent-round-1_4_0.R): the dataset
catalog carries the 24 NAPS air-quality keys and the CCHS 2022 PUMF, so both arms list the same 71
keys (morie_fetch_naps() fetches ECCC's hourly files; a zip with no member named yields its first
CSV); rmorie list-datasets prints a route for every key and the footer; rmorie pull,
inspect and verify without a path exit 2; exec sees the package's functions; edit refuses a
terminal editor when stdin is not a terminal; generate-template takes the module as a positional
argument, fills the module description, rejects an unknown module and never overwrites without
--force; emissions --seconds Inf is refused; verify-pollution validates its numeric flags, uses
one reference concentration throughout and counts avoided deaths over the exposed share; the ArcGIS
downloads show their progress; the synthetic-CPADS notice is given once per session; the launcher
written by install_cli() pins its library with .libPaths() so an R_LIBS in ~/.Renviron cannot
replace it; the liboqs message is one sentence; a login instruction always names the email route.
Round three from the same agent: verify fails p-values outside [0, 1] and reversed confidence intervals;
--module selects the module's own tables (directory or name prefix) and an unknown module is refused;
a stratum shorter than its allocation carries weight 1 under --per-stratum; run-modules/pipeline
reject an unknown module before loading data; run-module fails when a module wrote nothing and says
what the artifact modules collect; otis-analysis and mapq-psychometrics say they run on synthetic
frames; morie_run_pipeline() refuses a missing project_root; morie_bricklayer(check = TRUE) no
longer mistakes the R launcher for the rmorie-cli binary; morie_siu_sanity_check() accepts the
reviewed corpus's bare officer counts and reasoning text; HTML entities in SIU text are decoded
(', accented names); emissions --country names an unknown code; the OTIS causal grid returns an
empty table with a message; the fallback cause names a rejected GEMINI_API_KEY; a failed read says
why and what the Wayback Machine said; pull --out names a file it cannot write; sample --seed
must be a number; the morie_write_audit_markdown() example runs; hyphenated catalog keys (the NAPS
keys) resolve as written; emissions --seconds tops out at a day; the first-paper template reads as a
sentence and names the rmorie command; verify-pollution defaults the counterfactual to 10 for NO2 and
5.8 for PM2.5; the morie_cluster() examples label their cases (the UL1.2 warning stays, by the
standard); morie_bricklayer() and agent() no longer speak of a separate, proprietary rmorie-cli:
the command line is this package's launcher (install_cli()).
Fixes from the 1.3.9 stress test (every one with a test in test-stress-1_4_0.R): rmorie verify
no longer fails a table whose text column is blank throughout; inspect/verify --module
match the hyphenated module name against the tables it wrote; verify-pollution refuses
--exposure-mean without --exposure-prevalence instead of assuming prevalence 0;
emissions --seconds validates its value and the sampler thread is joined at exit (no more
core dump after an R error); sample --n is the total for stratified draws (--per-stratum
for N each) and the .weight column is announced and droppable (--no-weight); run-module
rejects an unknown name before loading data; exec prints what the code printed with a final
newline; agent --help and perseus --help describe the verb; login --to-email needs
--email; download-bootstrap asks for --survey instead of starting a 376 MB download;
analyze and explain use the same usage exit code; the tutorial says so when stdin is closed;
ask/percy name why no model answered (rejected hosted key, unreachable endpoint) and the
local fallback text carries a fallback attribute; install_cli() writes a launcher pinned to
the library it was installed from and honours ~/.Renviron; the duckdb banner is silenced on
listing verbs; morie_load_dataset(), morie_spillover_ht() and morie_run_pipeline() give
worded errors; morie_otis_causal_grid() skips a pair whose columns the frame lacks;
morie_install_extras() looks packages up without loading them; morie_emissions_track()
writes nothing unless output_dir is given; morie_matching_multi_treatment() skips the
reference level for integer treatments; morie_siu_sanity_check() accepts the corpus's yes/no,
true/false and lower-case gender values; the SIU LLM chain reads GEMINI_API_KEY and knows the
hosted tier; the README SIU calls match the functions; the crypto message names rmorie and
SystemRequirements lists liboqs. The 116 example topics that emitted R warnings now run clean
(corrected examples, and informational notes demoted from warnings to messages; morie_cluster() keeps its UL1.2 warning for unlabelled cases, by the standard).
Datasets: the fourteen Health Infobase tables (CPADS, CSADS, CSUS) download from
the portal and fall back to the copy at data.rmorie.com (hib/... keys); the three
OTIS research environments are fetched from data.rmorie.com as R objects
(morie_load_dataset("otisfin") returns the environment); the Ontario correctional
institution locations join the catalog as otisloc. The message for a missing
own file now says what it means.
The causal-estimators module runs again on the synthetic CPADS frame: its
AIPW line now lines the propensity rows up with the outcome rows by name
instead of predicting from a frame without the label covariates.
The hosted tier lists additional AI models (kimi-k2.6:cf, kimi-k2.7-code:cf, deepseek-v4-pro:cf, deepseek-v4-flash:cf, glm-5.2:cf, glm-5.3:cf, glm-5.3-flash:cf, gpt-oss-120b:cf, gpt-oss-20b:cf, llama-4-scout:cf, qwen3.8-27b:cf, nemotron-3-120b:cf and gemma-4-26b:cf) beside the
ollama.com ones, and falls back to them when a cloud model is rate
limited; rmorie ask --model gpt-oss-120b:cf picks one.
Downloads that drop part-way (a reset connection, a server that closes early) are started again, up to three attempts, with the reason on screen; a short body no longer passes as a complete file.
Command line, from a hostile QA pass: the launcher works under R 4.6 (which
keeps --args in commandArgs()); rmorie pull keeps what it fetches under
the user cache directory, so later run-module, pipeline and selftest
runs use the real PUMF; run-module without --output-dir writes under
morie-output/NAME; VERB --help describes the verb instead of running it;
verify checks CSV tables; ask exits 1 when no provider answered and names
a model nobody serves; login --token refuses a key the gateway rejects;
doctor distinguishes a rejected key from an unreachable gateway;
analyze exits 1 on an error payload; sample --n and missing files are
reported in words; edit no longer creates a file when there is no editor.
Statistics: morie_chi_square_test() gains correct = (Yates is now a
documented choice; Cramer's V uses the uncorrected statistic);
morie_linear_regression_ols() treats a constant column you supply as the
intercept and keeps standard errors aligned; morie_matching_nearest_neighbor()
warns that 1:1 matching without replacement estimates the ATT on the
matchable subset when controls are scarce (pass replace = TRUE);
morie_estimate_aipw() defaults to outcome_model = "auto" (logistic for a
binary outcome) in both arms; the causal-estimators module reports
influence-function and bootstrap standard errors instead of values about
165 times too small; AIPW, Kaplan-Meier and the E-value refuse missing or
invalid input in words. The package's JSON class is morie_json, so loading
jsonlite no longer prints an S3 override notice.
Every download draws a live progress bar (percent, size, rate) on a
terminal and prints milestone lines in a log: the catalog files, bootstrap
weights, StatCan tables, Wayback copies and the curated tables at
data.rmorie.com. options(morie.quiet = TRUE) silences it; the command line
turns it on, R sessions show it when interactive.
morie_hosted_manifest(), morie_hosted_datasets(),
morie_load_hosted_dataset(): the curated tables at data.rmorie.com,
opened by the stored MORIE key; morie_load_dataset(), morie_list_datasets()
and rmorie pull accept their db/table keys. rmorie pull --all
downloads the whole catalog; --dataset KEY on run-module,
run-modules and pipeline.
.cpads_default_csv() falls back to rmoriedata's synthetic CPADS PUMF when
the real one is not checked out; selftest skips only when neither exists; the
first-paper template ships; ASCII-only sources; lint-clean lines.
explain, inspect, verify, profile-dataset, sample, run-modules,
pipeline, emissions, verify-pollution, percy / agent, chat,
selftest, tutorial, generate-template, update, crypto, ingest,
download-bootstrap, exec, edit, percysuits. verify-earth-engine
says it lives on the Python side.morie_envhealth_crf_pm25() / _no2(), morie_envhealth_attributable_fraction(),
morie_envhealth_mortality_displaced(), morie_envhealth_burden(),
morie_envhealth_sensitivity(), morie_envhealth_equity(),
morie_envhealth_burden_by_fsa() and morie_verify_pollution(): the
Python morie.envhealth module, formula for formula, with the citations.morie_emissions_start() / _stop() / _track(): the CodeCarbon method
(TDP x utilisation + RAM, IEA grid intensity via
morie_emissions_carbon_intensity()), a C++ background thread sampling the
kernel's CPU counters, the same emissions.csv layout as the Python
package, and a bricklayer capsule per run (emissions_manifest.json plus a
signed capsule_bundle.json) that morie_emissions_verify() checks.
rmorie pipeline tracks and seals its runs.list-modules, run-module NAME [--output-dir DIR] [--cpads FILE], list-datasets, pull KEY [--out FILE.csv], cheatsheet, and
provider set --base-url URL --key KEY [--model NAME] | show | unset.
morie_module_names() lists the CPADS modules.morie_llm_provider_set(), morie_llm_provider_show() and
morie_llm_provider_unset() attach any OpenAI-compatible endpoint; the
setting lives in the credentials file shared with the Python package and the
provider chain and models use it. LLM_API_BASE_URL, LLM_API_KEY and
MORIE_API_MODEL take precedence.morie_load_dataset() example reads the built-in copy first, and the
Hawkes examples fit 400 and 500 events. research/ stays in the repository
and out of the tarball.MORIE_TEST_BUDGET_SECONDS) so the check fits
its 60-minute cap; every other platform runs the whole suite.morie_llm_hosted_models() returns the models the hosted tier offers
your key, with the default morie_llm_ask() would use as an attribute.rmorie models prints that list (default marked) and the local Ollama
server's models; rmorie doctor names them on its hosted line; rmorie ask --model NAME PROMPT picks one for a single call.morie_bayoptr() could stop with "non-finite value supplied by optim":
far from any improvement the expected-improvement closed form rounds to
a subnormal of either sign and optim's L-BFGS-B line search choked on
it. EI and PI are now exactly zero below the double minimum, and a
polish that fails leaves the unpolished candidate in the race instead
of ending the run. The documented example returns the same optimum as
the Python arm.morie_siu_index(lang = "en") and lang = "fr" returned an error: the
shipped siu_drid_manifest.csv.gz carried the column as X_language
(written through check.names = TRUE in an earlier build) while the
code reads _language. The shipped file is regenerated with the
rmoriedata header and the reader normalises the old name; the manifest
holds 2,532 English and 2,217 French reports.df (morie_siu_reports(), the
reviewed corpus from rmoriedata) before using it, quote the real
language counts, and say what the outputs-manifest example expects.morie_siu_audit_columns() drops blank and NA case numbers before
auditing: 628 of the 5,157 reviewed reports carry no parsed case number,
so sampling the corpus used to spend audit calls on empty strings.morie_llm_ask() without a reachable model now explains how to enable
one, including the hosted tier (morie_llm_login(), rmorie login),
instead of listing Python commands.R CMD check could not build it. The hosted-model fallback test
allows its (mocked) probe under R CMD check, which parks network
probes by default. 1.3.2 was tagged before its check finished and was
never submitted to CRAN; 1.3.3 is that release with the two fixes.morie_llm_ask() falls back to the MORIE hosted tier at
llm.rmorie.com (since retired) when no local Ollama answers. Sign in once with
morie_llm_login() (GitHub device flow), morie_llm_login(email = )
(a 6-digit code sent to the address), morie_llm_login(email = , to_email = TRUE) (the key itself mailed to the verified inbox) or
morie_llm_login(token = ) (a key obtained elsewhere, probed once).
morie_llm_logout() forgets it. The key lives owner-only in
$XDG_CONFIG_HOME/morie/credentials.json, shared with the Python
package; MORIE_HOSTED_KEY, MORIE_HOSTED_BASE_URL ("off" disables
the tier; an empty value also does on POSIX) and MORIE_HOSTED_MODEL
override it. When the gateway no longer lists the requested model the
first model it does list is used, so a model retired upstream cannot
turn every request into an error. morie_llm_probe_hosted() reports
the tier; the probes consult their cache before touching the network.morie_cli() dispatches login, logout, doctor, ask, analyze,
version and help; inst/bin/rmorie is the launcher and
install_cli() links it onto your PATH (an rmorie.cmd wrapper on
Windows). rmorie login --email [email protected] --to-email and
rmorie login --token mirror the R functions above.HypercubeQueue()) and the square-root law of patrol travel distance
(SquareRootLaw()).LateralRange(), SearchPod(),
OptimalCircularSearch()).ErlangC(), StaffingRequirement(),
ShiftSchedule(), ReliefFactor()).siu
CLI; SIU case-to-decision summaries are Kaplan-Meier; the Lakner
stock/flow measures in the MRM.emIRT::binIRT) and a
Philox Gibbs sampler for the CJR model; the W-NOMINATE fit and the
BLACKBOXT transpose scaling; majority tournaments (Condorcet, top cycle,
uncovered set, Banks set, Copeland, Borda), the exact yolk, agenda
setting, power indices, nucleolus and kernel; party competition
(Hotelling-Downs, valence, entry, bargaining, committees, PUNE),
voter utility and vote probability models, polarisation measures,
optimal cutting lines, the Gelman-King incumbency advantage, seat
allocation, issue salience and panel binary choice.did::aggte; synthetic DiD is synthdid's estimator; fuzzy DiD is
the Wald-DiD 2SLS; the wild cluster bootstrap follows boottest; RD
bandwidths, bias correction and the density test equal rdrobust and
rddensity; IV standard errors, GMM, Anderson-Rubin and IV probit match
ivreg; honest sensitivity is the Rambachan-Roth set; the Kitagawa
instrument-validity test and the CLR precision correction are real.boot::boot.ci intervals, DeLong's AUC interval, cox.zph,
concordance tie rules, Hommel and q-value pi0, influence measures, PPS
sampling, SMO support-vector machines, Little's MCAR test and Rubin
pooling, and upper-tail p-values taken from lower.tail = FALSE
throughout.research/LEDGER.md: the dark figure of
crime, contagion and branching ratios of self-exciting crime processes,
ecological correlation, the recording map behind downgrade-and-caution
rises, sentencing bounds under contamination, concentration, fairness
and deterrence) is implemented as morie_* functions with
Philox-seeded simulations that the Python arm replays exactly, and the
identification results are machine-checked in Lean 4 under
research/lean/ (176 theorems), next to the MRP proofs of the VSR
paper (38 theorems).Replaypack() writes its canonical text through a binary connection so
the bytes on disk equal the digested text on every platform. The
hosted-tier off switch works on Windows ("off"), where an empty
environment variable is dropped.An AST scan over the package found defaulted arguments never referenced
in their bodies. Those documented as no-ops are unchanged; the rest now
act: cramers_v() carries a noncentral chi-square interval at
confidence, random_effects_meta() implements method = "PM" and
"REML", link_test() fits model_type = "logistic",
morie_rdd_plot_data() returns the local fits (cutoff, p_local,
bandwidth, kernel), morie_rdd_sharp() bootstraps by cluster,
morie_rdd_bandwidth_rot() reports both sides of cutoff,
morie_weights_greg() iterates to max_iter,
morie_matching_att_matched() weights by the named column,
morie_matching_abadie_imbens_se() divides the reuse count by
n_matches, morie_brunner_munzel(), morie_iv_anderson_rubin(),
Landmark() and Aftfit() use alpha, the ingest helpers send
user_agent, morie_bayes_lm() honours stop_on_convergence, and
morie_spatial_voting_aldrich_mckelvey() refuses n_dims != 1 instead
of ignoring it.
mrm_twoprop_test() with no events (or nothing but events) in both arms
returns chi-square 0 with p = 1 and warns, instead of NaN from
chisq.test(). This is morie's own convention (the limit of two
identical arms), the same one the Python arm adopted in 1.3.2, so the
two arms agree again on degenerate tables.
morie_fairness_gini() is NA with a warning and an "undefined"
interpretation when a value is missing or non-finite (it returned 0,
"relatively evenly spread"). morie_tox_left_censor_impute() no longer
counts NA as below the detection limit: it stays NA, is left out of
fraction_censored, and is reported as n_missing. Same change in the
Python arm.
Every basicspace call (morie_spatial_voting_aldrich_mckelvey(),
morie_spatial_voting_blackbox(), morie_spatial_voting_bayesian_am())
is gated on a double matrix with at least two rows and two columns:
basicspace 0.25's Fortran writes past its arrays on a single-row or
single-column input and the process dies later (valgrind). Thin inputs
take the native path.
morie_spatial_voting_aldrich_mckelvey() reaches basicspace as documented:
it passed missing = NA, which basicspace rejects, so every call had
taken the fallback. Its numbers change: the stimulus positions are
now basicspace's, standardised to mean 0 and sd 1. On data with structure
to recover the two engines agree closely (stimuli correlate about 0.97,
intercepts 1.00) but not exactly; on structureless noise they do not
agree at all (0.04), as nothing constrains either. The respondent
intercepts and slopes are the per-respondent regression of the
placements on those positions on either engine, so they are defined for
every respondent; basicspace reports them only against a self-placement,
which this function does not take, and a 1-7 scale had given all-NA
respondent parameters. Non-finite stimuli from basicspace fall back to
the native solver.
predict.morie_cluster() computes the n x k distances to the centroids
directly. It built the full (n + k)^2 distance matrix and kept n x k of
it: 6 s and 2.8 GB at n = 10,000 for a 4 ms job, and out of memory at
n = 20,000. morie_cluster() gains iter_max for the k-means iteration
cap. One example and one test used R 4.4 idioms (%||%,
psmirnov(alternative =)) on a package that declares R >= 4.3.0.
Seven morie_ingest_a2aj_*() functions cover the A2AJ Canadian Legal
Data corpus (about 226,000 court and tribunal decisions and 27,000
statutes and regulations, maintained by Access to Algorithmic Justice at
Osgoode Hall and Toronto Metropolitan University): _coverage() lists
what the corpus holds, _search() and _fetch() wrap the REST API,
_download() and _load() bring one court's Parquet file from Hugging
Face into a data.frame, _citation_edges() turns the cited-cases field
into an edge list, and _gaps() tabulates the courts the corpus does not
carry. Every record carries its upstream_license.
Eight morie_ingest_canlii_*() functions wrap CanLII's REST API for
those gaps (the Human Rights Tribunal of Ontario, the BC Human Rights
Tribunal, the Ontario Superior Court, Alberta, Quebec and the remaining
provinces): databases, per-court decision lists with date filters, one
decision's metadata, the citator (cited cases, citing cases, cited
legislation), and the legislation browse; _case_id() derives CanLII's
identifiers from a neutral citation. A free key is read from
CANLII_API_KEY.
The native Parquet reader now decodes three-level LIST columns (one
vector per row, NULL for a null list), which is how the A2AJ citation
network fields are stored; other nesting is still refused rather than
silently mis-counted. The mrm-dataset-fetchers vignette gains a
section on both sources.
src/morie_dsp.cpp): the row offset into the covariance matrix was
an int product widened after the multiplication; it is now computed
in size_t.@examples
sections, so their Rd pages ran every example twice.Fifteen functions for studies whose target group is a small share of the frame, is reached by a design that is not simple random sampling, or is imperfectly identified by the frame itself. The three problems compound, and the package previously covered only the estimators, not the planning.
Sample size and design: morie_sample_size_proportion() composes the finite
population correction, a design effect and a non-response inflation in that
order; morie_sample_size_domain() answers the question the estimators could
not, namely how large the whole sample must be for the subgroup to reach a
stated precision, dividing by both the subgroup share of the frame and the
rate at which the frame identifies it. morie_deff_cluster() and
morie_neff_cluster() give the clustering component of the design effect,
which multiplies with the weight component from morie_design_effect().
morie_oversample_factor() returns the selection multiplier for a target
composition together with the weight-induced design effect it creates, so the
two are planned together rather than in sequence. morie_screen_design()
costs a two-phase screen for a rare group. morie_alloc_optimal() allocates
across strata with unequal per-unit costs, reducing to Neyman() when costs
are equal.
Weighting: morie_rake() performs iterative proportional fitting to known
margins, reporting convergence rather than assuming it.
Identification error: morie_misclass_correct() and morie_misclass_count()
implement the Rogan-Gladen correction, which recovers a prevalence from an
observed one when the classifier has known sensitivity and specificity. Where
group membership is recorded administratively rather than self-reported,
sensitivity below one biases every rate computed on the recorded counts.
Measurement equivalence: morie_dif_sample_size() sizes a differential item
functioning comparison, whose power is governed by the smaller group;
morie_dif_delta_mh() puts a Mantel-Haenszel odds ratio on the ETS delta
scale with its A/B/C classification; morie_invariance_compare() runs the
nested configural/metric/scalar sequence on chi-square, CFI and RMSEA;
morie_irt_theta_se() and morie_irt_marginal_reliability() report score
precision, which is worth reporting by group for the same reason DIF is.
New vignette subpopulation-design, including the governance framework the
arithmetic sits inside.
agent() quotes every argument it hands to the CLI. Unquoted, a
parenthesis in the task was a shell syntax error, a space split the task
in two and a ; ran the rest as a command. task, model and
backend are validated, and a stub binary on PATH now exercises the
branch the tests never reached.
morie_safe_relabel(), morie_decode_labelled(), morie_transfer_verify()
and morie_relabel_forensics(): relabel by name only, decode labelled
imports by code, verify an SPSS/Stata/SAS import against the source
program's code book and frequency table, and name the mechanical step
(alphabetical positional relabel, rotation, ...) that reproduces an
observed permutation. The OHRC 2023 correction's four-way rotation is the
documented case and the test fixture.
morie_marginals_verify() and morie_transfer_verify() gain strict;
with strict = FALSE a mismatch comes back as ok = FALSE with the
permutation and message instead of an error, so
morie_relabel_forensics() is reachable from the public path.
morie_audit_categories() now flags leading/trailing (incl. non-breaking)
whitespace, whitespace-variant duplicates, empty-string labels,
missing-value sentinels stored as labels, and a reference level that is
any of those. The identity permutation is reported as no permutation.
The shared C++ cor_pearson is centred and clamped to [-1, 1].
The greedy one-dimensional matcher refuses NA or NaN scores, a NaN caliper and a ratio below one instead of handing them to the sort, where a NaN comparator is undefined behaviour and corrupted the heap under the degenerate-input sweep.
Two more process-killers from the degenerate-input sweep are R errors
now: morie_kernel_pca() and morie_spectral_cluster() check the
feature matrix before kernlab's C code sees it (an empty list or a
zero-row matrix used to segfault), and morie_install_extras() refuses
empty or missing package names instead of handing them to
install.packages().
Sixteen modules that morie added in September and rmorie lacked now ship
here too, each checked against the Python arm and morie's R arm on
identical inputs to twelve digits with falsifiable anchors
(ledger/wave3/run3.sh): the bound families Bndcvr, Bndlgt,
Bndnmt, Bndsdo, Bnsipv; the clustered-design family Ccdsgn,
Ccmem, Ccngg; Cluseq, Cnsint, Crrcim, Cumcif, DepthP,
DepthS, Difpst and Rdmcbw. Five more come across without a Python
twin: morie_fairness_spatial_gan() and its family, the BPE tokenizer
(morie_tokenizer_new() and friends), the OTIS MAPQ study module, the
plot() method for morie_rich_result objects, and Qrf (quantile
forest pinball loss). torch and reticulate join Suggests as the
optional backends the fairness GAN and tokenizer can use.
Two hundred and thirty-six functions called set.seed() internally and
left the session's random stream replaced, so a user who had seeded their
session for reproducibility got identical downstream draws whatever seed
they had chosen. Every one of those sites now seeds for the rest of that
call only and restores the caller's stream (or its absence) on exit; the
seeded results themselves are unchanged. morie_det_rng() still seeds the
session, which is its documented purpose, but the twenty-six functions that
call it on their deterministic_seed path now restore the caller's stream
as well.
morie::core::mean() in src/morie_core.h, reached by morie_mean_cpp()
and by everything built on it (variance, standard deviation, moments,
bootstrap means), summed naively: rep(1e308, 3) gave Inf and
rep(1e120, 3) was off by 1.4e104, so the variance of three identical
values came back as 3e208. It now uses base R's algorithm (extended
precision sum plus one corrective pass) with a running mean as the
fallback when the sum overflows although every input is finite. Results on
ordinary data are bit-identical to mean(). morie_mean_cpp() and
morie_var_cpp() gain a shared argument: TRUE (the default) uses the
rmoriebricklayer kernel and FALSE the identical kernel rmorie vendors;
morie_mean() and morie_var() pick the vendored one whenever the
installed rmoriebricklayer predates 0.5.1, so the answer never depends on
which bricklayer build resolved.
The exact 1-D optimal matching kernel sorted its scores with <, which
is not a strict weak ordering when a score is NaN or Inf; under the
libstdc++ assertions that Fedora and Debian compile R packages with, that
aborted the whole R session. The kernel now refuses non-finite scores with
an error naming the element.
morie_cohens_d() warns when the denominator standard deviation is zero
or missing instead of returning Inf or NaN silently, and
morie_bootstrap_ci() warns when x has fewer than two observations.
morie_otis_stock_flow();
the exported name is mrm_otis_stock_flow().ByteCompile: no. Byte-compiling the package at install time took half of
the install (about four minutes on a laptop); R compiles functions on first
use anyway, so nothing is lost at run time.
Eighty-eight functions carried full roxygen documentation, references and
examples but no @export, so their examples reached them through
rmorie:::. Most are the genomic-prediction and kernel-method family
(morie_gblup(), morie_kernel_matrix(), morie_msm_*(), morie_fda_*()),
the AlphaFold loss functions (Alfviol() and its siblings) and
morie_laplc(). All are exported now and their examples call them
directly.
DESCRIPTION lists the pkgdown site URL, which pkgdown::check_pkgdown()
requires.
The MRM OTIS family gained the Lakner (1976) decomposition: average daily
population as a stock (sum(x_i)/t, days per day), average length of stay
as a flow (sum(x_i)/N, days per person), and the identity that ties them
(adp = N_a * alos / t). Flow and stock rates can carry opposite signs
over the same period, so reporting one as if it were the other reverses the
finding -- it did for an OTIS headline before this landed.
tests/testthat/setup.R decided whether it was running under R CMD check
by reading R_TESTS. R sources that file at startup and then unsets it, so
the variable is always empty by the time the setup file runs: every
skip_on_cran() in the package was inert, on CRAN included. The
discriminator is now _R_CHECK_PACKAGE_NAME_, which survives. With the
gates actually firing, the tests stage dropped from 956s to 192s and the
whole check from over an hour to about 39 minutes -- which is what brought
it back inside r-universe's sixty-minute budget.
.siu_resolve_drid() and its fetch path reached the live index on every
check run. They now return NA_integer_ unless live fetches are explicitly
allowed, so a check no longer depends on a third party being up.
.gh_pairwise_sq() built its Gram matrix from a BLAS call whose result is
only symmetric up to rounding, and the Cholesky downstream rejected it on
some platforms. The matrix is now symmetrized explicitly. morie_ghgpm()
also checks finiteness first and preserves the real chol() error instead
of replacing it.
rmoriebricklayer (>= 0.4.8) is required for the native decimal
conversion; older copies are skipped rather than failed.
digest::digest2int() hashes a plain char, whose signedness is the
platform's: signed on x86, unsigned on arm. The native kernel forced
signed char, so it agreed with digest on x86 and diverged on arm for
any byte above 0x7f -- the r-universe arm64 builds failed on "é",
expecting -1366949649 and getting 26298542. The cast now follows the
platform, as digest does.
R CMD Rd2pdf failed on 1.2.1's sources, which is what the CRAN check
reported as "checking PDF version of manual without index ... ERROR".
Three causes, none of them visible without running the build:
\code{} Rd reads an apostrophe as opening an R-style quoted
string, so a transpose or a derivative (V', g') swallowed the
closing brace and \code never terminated. 90 spans in man/ and 42
in roxygen are now escaped as \', which renders identically.\[ and \] are not Rd escapes; inside \eqn/\deqn they reached
LaTeX as display-math delimiters nested in display math, and elsewhere
as undefined control sequences in the HTML manual. 661 occurrences.\u(t) for \mu(t); \u is LaTeX's
breve accent. RFC 7464's record separator was a literal control byte
that had decayed to U+FFFD, which pdflatex cannot typeset.Also removed: a data() call that read as a load into the global
environment, a stray top-level 5.rds, and literal angle-bracket
placeholders that reached the HTML manual as unrecognised tags.
inst/ went from 2.18 MB to 0.25 MB, and the tarball from 11.66 MB to
9.73 MB: 54 fixtures byte-identical to rmoriedata's are resolved from
there at runtime, the generated dataset catalogue is built on demand,
and describe_corpus.Rds moved to rmoriedata with the rest of the data.
CRAN's rule is that \dontrun marks an example the user cannot run --
missing software, an API key -- and that everything else belongs in
\donttest, because \dontrun prints a "# Not run:" warning that
misleads the reader. 157 example blocks were wrapped in \dontrun purely
because they reach the network, which is not a reason CRAN accepts: they
are runnable, they are already guarded with try(), and they degrade
cleanly offline. They are now \donttest.
\dontrun is kept in the handful of places it is correct: the Vertex
token/Gemini calls, the StatCan and NIBRS ingests that need an API key,
and install_extras(), which installs packages.
CI keeps its speed through _R_CHECK_DONTTEST_EXAMPLES_: false rather
than through misleading markup -- --as-cran executes \donttest, and
on a GitHub runner the network examples crawl against throttled services
until the job times out. CRAN's own machines decide for themselves; the
full example sweep runs on L14.
This release merges the feat/native-specializations line (650 commits).
Every estimator that previously delegated to a heavy runtime dependency
now has a native implementation, cross-validated against the package it
replaces and benchmarked: double-ML runs 40-61x faster than DoubleML,
the R-learner causal forest 2.9x faster than grf, optimal matching
beats optmatch's own optimum, and design-based GLM matches svyglm to
1e-6. The full matrix is green on Windows, macOS, Ubuntu (release,
devel, oldrel-1) and a true Debian bookworm container.
morie_jsonlt_* is now jsonlite's whole public surface, written in base
R and pinned to jsonlite byte-for-byte by tests/testthat/test-jsonlt-parity.R
whenever jsonlite is installed (1,742 grid cases, 0 differences): every
toJSON() option including rownames, keep_vec_names, json_verbatim,
always_decimal, time_format, UTC, no_dots; jsonlite's number
formatting (num_to_char / modp_dtoa2 rounding, digits = NA and
I(n)); the fromJSON() simplifier (record lists, matrices and
higher arrays, $date, _row, "NA"-string vectors, bigint_as_char);
yajl-layout prettify() / minify(); validate(); serializeJSON() /
unserializeJSON() for every storage mode; base64_enc/dec (+url);
read_json / write_json / parse_json; ndjson stream_in /
stream_out; rbind_pages..morie_from_json,
.s03json_*), so no code path needs jsonlite at run time; jsonlite stays
in Suggests for the parity test only.morie_digest(), morie_hmac(), morie_make_raw(), morie_digest2int(),
morie_sha1() (with every class method) and morie_aes() reproduce the
digest package's API and bytes: MD5, SHA-1, SHA-224/256/384/512, CRC-32,
CRC-32C, xxHash32/64, MurmurHash3 x86_32 and Jenkins one-at-a-time in
src/morie_digest.cpp, written from their specifications; AES-128/192/256
in ECB/CBC/CFB/CTR. serialize() header skipping, length/skip
windows, raw = TRUE, seed, file = TRUE and the error modes follow
digest exactly. tests/testthat/test-digest-parity.R checks the
published test vectors always and every routine against digest itself
when it is installed. spookyhash, blake3 and xxh3 are not provided (no
specification to implement from) and say so when requested.man/ and NAMESPACE are generated from the R/ roxygen again. The
docs marathon had written about 1,950 \examples{} blocks straight into
.Rd files and hand-appended exports to NAMESPACE; both were lifted
back into the sources (@examples on the function's block, exports
only from @export), so roxygenise() reproduces the tree exactly.#' line between two expressions). They are re-attached to
the functions they document (morie_estimate_aipw(), morie_mtr2sx(),
morie_haldane(), .fit_propensity() and others); the stray export
names that the merge produced are gone.R/rcpp_docs.R
(documentation-only stubs); R/RcppExports.R is pure generated glue
again, so Rcpp::compileAttributes() no longer deletes any docs.rmorie no longer pulls in 'jsonlite' or 'digest' through
rmoriebricklayer: rmoriebricklayer 0.3.8 hashes with its compiled
SHA-256 core and reads/writes JSON with a native codec. The stale
requireNamespace("jsonlite") guard in morie_fetch_tps() is removed.R/ now has roxygen, an .Rd page and a
\value; 0 unparseable pages, 0 checkRd findings. @param text is
derived from the body and quotes its evidence. Executable \examples
were added in execution-gated batches; network loaders use
\donttest{} with real arguments. Gaps that remain are counted, not
hidden.changepoint_rjmcmc(): the uniform stream returned the same number
forever (rmorie), the birth/death slices used descending colons at
k = 0, the interval search stopped one edge short, and the reverse-move
probability indexed one entry low (log 0, births never accepted). The
sampler is now byte-identical across the two R arms and mirrors the
Python arm: same stream, same k posterior.
morie_survnnr_fit(): backprop took an outer product against a nested
list; the input activation was stored as list(list(a)).
mqtmpl method = "em": was a Haley-Knott regression on a genotype
probability that ignored each individual's flanking markers (every row
identical, LOD all NA). Now the Lander-Botstein EM the Python arm
implements, per-individual backcross probabilities, cM positions;
the three arms agree to 1e-10 on a shared anchor.
Native Parquet reader: an OPTIONAL BYTE_ARRAY column read its own
definition levels as its first string ("d\001") and dropped its last
row; the schema's repetition type now decides and 1-bit RLE/bit-packed
levels are decoded and spliced (nulls become NA).
VERSION_INVENTORY.csv regenerated (the drift check would have failed).
morie_estimate_plr() reported its method as "(morie native)" inside rmorie; it is "(rmorie native)".
v-prefixed vector IDs with a clean error for
non-numeric input; morie_datasets_statcan_full_csv_url() reports
status=NULL on HTML outage pages instead of crashing..morie_tps_parse_datetime(): explicit
per-format passes (ISO 8601 incl. T/millis/Z, US m/d/Y with and
without AM/PM). Fixes mixed-format vectors losing ISO entries
(as.POSIXct with no format throws when any element fails, R >= 4.3)
and the sample-CSV crash on Debian..morie_dataset_http_post_json(): non-JSON service error pages raise
a clear error with a response snippet.optmatch added to Suggests (used via @examplesIf in the
full-matching example), network examples wrapped in try(), long
example lines wrapped, non-standard top-level files .Rbuildignored.skip_if_not_installed(); live SIU network tests are opt-in
(RMORIE_NETWORK_TESTS=1) and skipped on CRAN.r-minimal-deps CI job permanently guards the Suggests-less
install path.fwildclusterboot from Suggests and removed the
.morie_did_have_fwildboot() helper + the
if (.morie_did_have_fwildboot()) { fwildclusterboot::boottest(...) }
branch in morie_did_wild_cluster_bootstrap(). The function
now goes straight to the base-R Rademacher/Webb wild-cluster
bootstrap (which already existed as the fallback and mirrors
the Python implementation; no math change for any caller).s3alfisc/fwildclusterboot
and 3MMM.40b added s3alfisc/summclust, but the resolver still
reported summclust: Can't find package called summclust -- so
the recursive-Remote pattern is structurally fragile. Following
the same "drop optional CRAN-archived/GitHub-only deps" pattern
used for rdd in 3MMM.40.Remotes: now lists only synth-inference/synthdid, which has
no GitHub-only transitive Imports.s3alfisc/summclust to DESCRIPTION Remotes:.
fwildclusterboot Imports summclust, which is also
GitHub-only (never on CRAN). 3MMM.40 added the
fwildclusterboot Remote but pak's recursive resolver still
failed one level deeper because it does not auto-recurse
through a Remote's own DESCRIPTION. summclust's Imports
(utils, dreamerr, MASS, collapse, generics, cli, rlang) are
all on CRAN, so the chain terminates here.R/datasets.R (6 sites): dropped the invalid n = 2L argument
from strsplit(). Base R's strsplit() has no n=; the call
silently ignored it on most R versions but errors on R-devel.R/dataset_load_by_key.R: removed the spurious
max_features = max_features argument from the
morie_datasets_ontario_ckan_by_key() dispatch. That function's
formals are only (dataset_key, offline, resource_id) -- passing
the unused formal caused a hard error in the dispatcher example.R/ingest_statcan.R: replaced the non-existent
cansim::set_cansim_api_key(api_key) call with the documented
mechanism. cansim has no such helper in any current CRAN
release; it reads CANSIM_API_KEY from the environment.
morie_ingest_statcan_cansim() now mirrors a user-supplied
STATCAN_API_KEY into CANSIM_API_KEY when only the morie alias
is set.R/spatial_voting.R::mlsmu6: added is.finite(prev_stress) guard
for the convergence check. prev_stress starts as Inf, so
iter 1's abs(Inf - stress) / max(Inf, 1e-12) = NaN triggered
"missing value where TRUE/FALSE needed" and broke the
\examples{} block. The first iteration now skips the
convergence check cleanly; iter 2+ uses real values.morie_dataset_portal_catalog()
(only the @export tag was present; the docstring upstream was
attached to the sibling _clear_cache helper).man/morie_dataset_portal_catalog.Rd and
man/morie_entheo_clone_dmt_imaging.Rd regenerated.rdd from Suggests. CRAN archived it in 2024 and pak
could no longer resolve it. The only morie callsite
(morie_rdd_mccrary()) used it as a fallback when rddensity
wasn't installed; rddensity itself is on CRAN and in Suggests,
so the rdd branch was effectively dead code in any realistic
configuration.Remotes: s3alfisc/fwildclusterboot, synth-inference/synthdid
so pak can fetch the two remaining GitHub-only Suggests when
building the lockfile. Both upstream repositories are verified
live (HTTP 200 from api.github.com/repos/...).Phase 3LLL reverses the destructive \eqn{LATEX} -> \code{LATEX} swap
shipped in commit f399ec41a (Phase 3KKK1+2). That swap eliminated the
"Lost-braces" warning but at the cost of stripping LaTeX math
typesetting from the PDF/HTML manual and turning every greek letter,
\hat, \sum, \frac, etc. into an "unknown macro" warning at
R CMD check.
The proper Rd-compliant fix is the two-argument form:
\eqn{LATEX}{ASCII fallback} # inline
\deqn{LATEX}{ASCII fallback} # display
Every affected line (104 R files) now uses this form, preserving PDF
math while satisfying the Rd parser. Driven by fix_rd_math.py, a
LaTeX->ASCII transformer covering the common Greek alphabet,
operators (\sum, \int, \hat, \bar, \frac, \sqrt), and
relation symbols.
New morie_install_extras() lets users install the ~50 optional
Suggests: packages in one call. CRAN policy forbids
install.packages() at .onLoad() time, so morie ships an opt-in
helper instead. Three modes:
morie_install_extras() # missing only (default)
morie_install_extras("all", ask = FALSE) # everything, CI-safe
morie_install_extras(c("hawkes", "sf")) # named subset
The helper also probes for the C system libraries libcurl,
libsodium, and liboqs and prints platform-specific install hints
when any are missing. System libraries must be installed BEFORE
re-installing morie so the configure-time probes link the C/C++
backends against them.
Cross-portal morie_dataset_portal_catalog() grows from ~1,044
rows to 9,242 rows across 14 portals. Every Socrata / CKAN /
ArcGIS Hub / Opendatasoft portal morie touches now has its full
public catalog integrated offline.
startindex= pagination, not Socrata offset=).morie_datasets_nyc_socrata_by_id() +
morie_datasets_chicago_socrata_by_id() (mirror the 3FFF3
Calgary/Edmonton pattern). morie_datasets_load_by_key() routes
chicago + nyc_opendata sources through them; max_features now
threads as the SODA $limit.calgary_opendata 933 nyc_opendata 2861
chicago 1864 ontario_ckan 38
edmonton_opendata 2027 ottawa_opendata 287
montreal_opendata 401 statcan_ccjs 10
nyc_nypd 8 toronto_opendata 540
tps_arcgis_hub 71 tps_psdp 11
vancouver_opendata 190 vpd_geodash 1
-------
9242
Integrated fixture footprint: ~3.4 MB of catalog metadata; per-row
unwound this is the metadata equivalent of every NYC dataset
descriptor + every CKAN package summary + every Hub item -- offline
queryable via morie_datasets_browse(keyword=...).
Major sprint adding 14 open-data portals + a unified browse/load
interface. The cross-portal morie_dataset_portal_catalog()
now spans 9 cities + 1 federal source + ~800 dataset entries
across 4 different API protocols.
law_code resolver. New
morie_datasets_nyc_nypd_law_books() (46-row statute book ->
human name + jurisdiction dict; PL, VTL, CPL, ABC, AC, COR, AM,
PHL, ED, GB, GCI, HTH, PAR, LOC, FOA, RR, TAX, RPA, RP, PRL,
TWN, ...) + morie_parse_nypd_law_code() vectorised regex
parser. Added as 4th resolver in
morie_datasets_nyc_nypd_resolved().morie_datasets_nyc_boundaries_catalog()
unified index.morie_datasets_tps_psdp_resolved()) -- division + hood158 +
hood140 + NIA + psdp_class 5-way join, mirrors the Chicago /
NYPD _resolved() patterns. Plus
morie_datasets_tps_police_divisions() (16 post-amalgamation
TPS divisions).morie_dataset_portal_catalog() -- 7
initial portals, 336 datasets, uniform schema (dataset_key,
source, id, api_modes, loader, dict_url, n_rows_bundled). Added
Vancouver Open Data (Opendatasoft v2.1, 190 datasets). Folded
SODA3-auth note into the SODA3 helper docstring per
Socrata support article 34730618169623.zip_path = mode
for the full 915k-row feed.morie_datasets_statcan_cube_metadata() +
morie_datasets_statcan_vectors() +
morie_datasets_statcan_full_csv_url() wrappers.morie_datasets_browse() + morie_datasets_summary()
-- filter the cross-portal catalog by keyword / portal /
api_mode / loader regex with AND-composable predicates.morie_datasets_load_by_key() -- single dispatcher
resolving any catalog dataset_key to its loader across all
portals.package_show -> first-CSV resource
auto-resolution. MTL + TO generic CKAN keys now Just Work
through morie_datasets_load_by_key().mode = c("auto","soda2","soda3","odata") +
app_token args on the dispatcher; routes through SODA3 for
Socrata-backed sources, silently ignored elsewhere.chicago 8 ontario_ckan 38
nyc_nypd 8 vancouver_opendata 190
nyc_opendata 10 vpd_geodash 1
tps_arcgis_hub 71 statcan_ccjs 10
tps_psdp 11 montreal_opendata 23
toronto_opendata 208
calgary_opendata 157
edmonton_opendata 195
ottawa_opendata 106
Total ~ 1044 catalog rows.
Formula corrections (affect Python AND R sibling identically):
iv.morie_iv_wald / iv.wald_estimator Wald-LATE delta-method SE
previously omitted the Cov(num, den) term, biasing the SE under
realistic Y-D correlation. Now includes - 2*(num/den^3) * cov(y, d) / n per-stratum aggregation.dsp_waveform.morie_dsp_higuchi_fd / _waveform.higuchi_fd
fractal dimension previously summed M-1 differences instead of M
(Higuchi 1988 eq 1 specifies floor((N-m)/k) summands). Fixed by
using M+1 indices so diff() yields M terms.R-side feature additions:
src/morie_hawkes.cpp,
morie_dsp.cpp, morie_matching.cpp, morie_spatial.cpp)
exposing 14 // [[Rcpp::export]] symbols.R/{tps_hawkes_advanced,dsp_filters,matching, spatial_voting}.R now dispatch to the C++ kernels when the
compiled .so is loaded, falling back to pure-R otherwise.LinkingTo: Rcpp, RcppArmadillo (was: Rcpp).Other fixes carried from the 5-layer review on 2026-05-22 (all Python-parity-verified before applying):
R/survival.R .validate_te now returns ok mask; KM/HR/concordance
callers re-align group/risk_score by mask instead of seq_along.R/iv.R JIVE projects only the endogenous columns (was: every
column including intercept and exogenous controls), matching
src/morie/iv.py:1604-1613.R/did.R morie_did_aggregate_gt_att SE uses k = cell count
(was: nrow(g), equivalent only when nrow(g)==1).R/did.R morie_did_test_parallel_trends returns
joint_chi2 + joint_df, keeps joint_f_stat as alias.R/inference.R Clopper-Pearson exact CI handles successes==0
and successes==n edges instead of calling qbeta(., 0, .).R/weights.R morie_weights_brr warns on odd-size strata.R/spatial_voting.R Hare 2018 + King 2003 citation corrections.R/tps_statphysics.R Helbing 2010 venue corrected (NJP not PNAS).Earlier from 2026-05-22 marathon (already in 0.9.5.6 in tree):
y[,"status"] not scalar nevent.((n_h-1)/n_h)*sum_{i in h} diffs_sq_i.r = Z/sqrt(n1+n2) (was: n1*n2).n_effective_tests sums fractional part for all eigenvalues.n_treated^2.R-side describe() parity closure. Patch release that closes one
of the two parity gaps named in v0.9.5.4: the pedagogical
narratives that the Python sibling exposes via morie.describe()
are now available on the R side via morie_describe() and the
string-only variant morie_describe_by_name().
R API additions:
morie_describe(callable) — takes a function object OR a
character scalar (with or without the morie_ prefix). Prints
the pedagogical narrative for the named callable.morie_describe_by_name(name) — string-only variant.Integrated data:
inst/extdata/describe_corpus.Rds — a single xz-compressed Rds
(~1.6 MB on disk) containing 36,433 named character entries.
Names are the callable mnemonics (the 4-7 character forms);
values are the markdown narrative bodies sourced from
src/morie/fn/describe_<name>.md. The Rds is loaded once per
session and cached in a package-private environment.Build tooling:
tools/capsule-describe-files.R — re-runs the Python-to-R sync
when src/morie/fn/describe_*.md changes. Run from the repo
root with Rscript tools/capsule-describe-files.R.Tests:
tests/testthat/test-describe.R — 17 tests covering lookup,
prefix stripping, .md extension stripping, unknown-name
diagnostics, type-rejection, function-object capture via
substitute(), and cache identity across calls. All pass on
the development build.Remaining parity gap:
morie.crypto educational primitives ship on the Python side
only; a native R + Rcpp port (ML-KEM, Dilithium, NTRU,
McEliece, ECC, hybrid PQC) is planned for v1.0.0. Calling into
the Python side via reticulate is not added in v0.9.5.5; the
scope was set at the native-R port path, which is a larger
arc and the natural place for a v1.0.0 milestone.Doob → MRM chi-square rename. Patch release with deprecation aliases; no breaking changes for existing user code.
Naming:
papers/ is preserved; the src/morie/sprott_doob.py and
src/morie/doob_trends.py author-named modules are also
preserved.Python API (with deprecation aliases):
morie.otis_all_analyze.analyze_c_doob_chi2() -> analyze_c_chi2()
morie.otis_all_analyze.analyze_d_doob_chi2() -> analyze_d_chi2()
Old names still work but emit DeprecationWarning. They will
be removed in a future release; update callers at your
convenience.
R side: no R API changes; the R chi-square family was already
renamed in v0.9.5 (vignette chi-square-and-anova.Rmd).
Patch release over 0.9.5.2.
pkgload in Suggests:. The pkgload skip-guard added
in 0.9.5.2's test-cov-fallbacks.R used pkgload::dev_packages()
without declaring the package in DESCRIPTION's Suggests:,
producing a '::' or ':::' import not declared from: 'pkgload'
WARNING under R CMD check. No user-visible functional change;
the warning is informational, but it should not have shipped in
0.9.5.2.morie_siu_sanity_check's description
used date_*_iso and number_of_* as inline text, which
roxygen2's markdown mode rendered as nested \emph{\emph{...}}
in the generated Rd and as nested <em> in the HTML manual.
win-builder flagged this as an HTML validation NOTE. Wrapping
the identifiers in backticks (now rendered as \verb{...})
resolves it.CRAN Policy: full cache-leak fix (supersedes 0.9.5 which was uploaded to win-builder with incomplete cache-isolation).
morie_db_connect() default cache-dir flipped from
tools::R_user_dir("morie", "cache") to a session-scoped
tempdir() subdirectory; matches the convention already set
for morie_fetch_siu() and morie_fetch_tps() in 0.9.5. Now
no morie function writes outside tempdir() unless the user
explicitly opts in by passing db_path = morie_cache_dir(...)
or cache_dir = morie_cache_dir(...).morie_cache_clear(subdir, confirm) user-facing function
for actively-managing the persistent cache (CRAN Policy
requirement for R_user_dir caches).morie_cache_dir(subdir) is now exported with a subdir
argument so users can compose per-subsystem persistent paths.morie_cache_* examples (store, load, list) now use
explicit db_path = tempfile() so R CMD check never writes
outside tempdir().morie_check_plugin_license error-path example moved from
\donttest{} to \dontrun{} (intentionally errors when
passed an incompatible SPDX).morie_fetch placeholder-URL example moved from
\donttest{} to \dontrun{} (example.org doesn't host CSV;
the URL is a documentation placeholder).crimsl.utoronto.ca references in R/mandela.R and
R/rmorie-package.R rewritten as plain-text references; the
U of T web server returns 403 to win-builder's IP even though
the URLs are publicly reachable from browsers.inst/WORDLIST listing real technical terms (AIPW, ATC,
ATT, CATE, Hawkes, MRM, etc.) so the win-builder
spell-checker no longer flags them.Documentation + CI hardening (added 2026-05-21 to the v0.9.5 release branch alongside the SIU + rename work):
vignettes/siu-pipeline.Rmd) — end-to-end
walkthrough of morie_fetch_siu(), morie_siu_audit_case(),
morie_siu_anomaly_check(), morie_siu_compare(),
morie_siu_llm_extract(), morie_siu_translate(), and the
canonical-override system. 14 vignettes total now.vignettes/chi-square-and-anova.Rmd
previously called the MRM chi-square family the "Doob $\chi^{2}$
family", which incorrectly singled out one of the three named
authors (Sprott, Doob, Iftene) of the source contingency tables.
Renamed to "MRM chi-square family". The Sprott / Doob / Iftene
author citation to the source tables is unchanged._pkgdown.yml shipped — a minimal pkgdown configuration so
contributors can build a local documentation site with
pkgdown::build_site(). The file is .Rbuildignored so it
doesn't ship in the CRAN tarball.inconsolata LaTeX font installed.
pkgcheck's internal rcmdcheck builds the PDF manual, which
needs inconsolata.sty. Without it pkgcheck reported a spurious
"R CMD check found 1 warning" against a package that has 0
warnings in the dedicated r-cmd-check.yml matrix. The
pkgcheck job now installs tinytex + inconsolata before running.lintr / goodpractice cleanups:
T_horizon instead
of T for the time-horizon parameter, so the auto-generated
R/RcppExports.R no longer trips R linters that flag T as a
potential TRUE shadow. The math convention is preserved in the
C++ docstrings; only the parameter NAME changed.setwd() in morie_run_workflow_step() replaced with
withr::local_dir() (goodpractice no-setwd linter).morie_* prefix so they
no longer collide with same-named functions in other CRAN
packages. Examples: chi_square_test → morie_chi_square_test,
kmeans_clustering → morie_kmeans_clustering, etc. Names that
were already morie-specific cryptic abbreviations (agset, brdgr,
fzhdc, …) are unchanged.SIU harvester: polite by default, manifest-aware, retry-aware, and auditable against the original published reports.
Persistent HTML cache + per-case audit. morie_fetch_siu(cache_html = TRUE) saves every fetched report and news-release page under
<cache_dir>/html/ (gzipped, ~80-100 MB for a full sweep). The
saved HTML is the canonical ground truth for every row in the
emitted CSV: any later question of the form "did the parser get
this field right?" is decidable by reading the cached page for
that case. morie_siu_audit_case(case_number) returns the
parser's 1-row data frame, the raw report and news HTML, and
HTML-stripped plain text for both, all from cache when available.
morie_siu_compare() — line up the parser's output for a
case against a user-supplied external table (column map and case
key are caller-controlled) and show the surrounding report HTML
excerpt for each disagreement. No external source is treated as
authoritative; the function exists so the user can adjudicate
parser-vs-external mismatches against the actual published
report. The published report HTML is the only ground truth morie
recognises for SIU fields.
Free by default. The LLM helpers now default to \code{model = c("ollama", "gemini")} -- a free local Ollama model first, with paid Gemini as fallback only if Ollama is unavailable. Users who install Ollama and pull a free Gemma / Qwen / Llama / Functiongemma variant (\code{ollama pull gemma3:4b}) get the full second-coder / audit / anomaly-check stack at $0 ongoing cost. \code{OLLAMA_HOST} defaults to \code{http://localhost:11434} when unset, so the zero-config path is just "install ollama, pull a model, done".
AI second-coder (Gemini / Claude / Ollama).
morie_siu_llm_extract(case_number, model = "gemini") sends the
cached report HTML through a large-language-model endpoint and
returns the same 64-column row format as the C++ parser, so it
drops straight into morie_siu_compare(external = ...) for an
independent diff. model accepts a character vector for
fail-over, e.g. c("gemini", "ollama") uses the paid Gemini
endpoint when available and silently falls back to a local /
free Ollama-compatible model otherwise. Credentials are read
from GOOGLE_API_KEY / ANTHROPIC_API_KEY / OLLAMA_HOST;
nothing is hard-coded.
morie_siu_translate_fr_to_en() — self-improving SIU.
For SIU cases that exist only in French (no English-language
paired drid; ~1-2 per year of SIU output), translate the
narrative_summary, news_release_summary, news_release_title and
relevant_legislation into English via a local Ollama model
(default $0 cost, no API key needed) and persist each
translation as a canonical override via
\code{morie_siu_record_correction()}. Idempotent (skips
already-translated cases) and self-improving (every run leaves
morie better at returning English content for French-only
reports). Maintainers can promote the resulting overrides into
the shipped \code{inst/extdata/siu_canonical_overrides.csv.gz}
so all users get the English text on the next package update.
French police-service acronyms. The modal-service detector now also recognises SPT (Service de Police de Toronto), PPO (Police provinciale de l'Ontario), SPRH (Halton), SPRY (York), SPRP (Peel), SPRD (Durham), SPRN (Niagara), SPRW (Waterloo), SPO (Ottawa), SPL (London), SPH (Hamilton), SPW (Windsor), SPG (Guelph), SPK (Kingston) and maps each to the canonical English name. Closes the remaining French-only-case gap; 12-TFD-104 in the 2012 corpus now reports \code{Toronto Police Service} correctly.
99.955% format-clean on the full 2,218-case corpus.
Empirical measurement via morie_siu_sanity_check() on the
freshly-harvested SIU.csv: 2,217 / 2,218 rows have zero format
issues; the lone remaining case is a 2012 French-only report
(12-TFD-104) without an English-paired drid. The earlier
95.45% baseline ate four further fixes: (a) Unicode apostrophe
/ quote / dash normalisation in lower_ascii() so the title-
finder matches "Director's report" (U+2019) cleanly, (b)
"Overview" as a section_4 fallback for 2014 reports that
retitled "The Investigation", (c) French "L'enquête" / "Aperçu"
fallbacks for French-only reports, (d) full SIU police-service
acronym table (OPP, TPS, HRPS, NRPS, PRP, YRP, DRPS, WRPS, OPS,
LPS, WPS, GPS, KPS, BPS, BPPS, CKPS, PRPS, GSPS, SSMPS, SLPS,
SPS, TBPS, BPSB) -- old reports use the acronym throughout and
never spell out "Ontario Provincial Police", and the modal-
service detector now picks up "OPP" → "Ontario Provincial
Police" automatically.
Interleaved report + news fetch. morie_fetch_siu() no longer
walks the corpus in two strict phases (fetch all reports, then
fetch all news). It now uses a rolling-window batched fetcher:
each batch of 250 reports fires in the same rate-limited pool
as the previous batch's news pages. While the next 250 reports
are downloading, the news pages for the nrids we just parsed
are downloading alongside. Roughly halves cold-start corpus
wall time (~30 min instead of ~58 min on the full 4,700-drid
sweep) without changing the per-second rate the SIU site sees.
Canonical overrides — the parser LEARNS from corrections. Every verified \code{(case_number, field, value)} tuple recorded via \code{morie_siu_record_correction()} is applied to \code{morie_fetch_siu()}'s output on subsequent runs. The shipped \code{inst/extdata/siu_canonical_overrides.csv.gz} holds the maintainer-confirmed table (starts empty in v0.9.5, populated by the LLM-audit + human-review workflow over time). The user-side \code{<cache_dir>/canonical_overrides.csv} merges in too -- users can fix their local copy without touching the package source. This is morie's "memory": wrong cells get found via \code{morie_siu_sanity_check()} or \code{morie_siu_audit_columns()}, corrected once, and the fix propagates to all users on the next package update -- no C++ rebuild needed.
morie_siu_sanity_check() — fast format-validity pass over
every row of an emitted SIU table. Flags case_number that
doesn't look like an SIU id, date_iso that isn't ISO 8601,
number_of that isn't a positive integer, charges_recommended
that isn't "Yes"/"No", page-chrome strings leaked into
narrative_summary or other content fields, etc. Returns a data
frame ordered worst-first so maintainers can pop the cached
HTML for any flagged row and adjudicate. Runs in milliseconds,
no network, no LLM, no API key required.
morie_siu_audit_columns() — closed-loop per-column accuracy
audit. Runs the anomaly check across many cases and aggregates
by field, returning a data frame sorted by agreement rate
(worst first) so maintainers can prioritise which regex
extraction pattern to fix next. Concrete disagreement examples
for each field are attached as the \code{"examples"} attribute.
With \code{model = "ollama"} pointed at a local Gemma / Qwen /
DeepSeek instance the audit costs zero API spend; chain
\code{c("gemini", "ollama")} for paid-first / free-fallback.
morie_siu_anomaly_check() — per-field "does the report
support this extraction?" audit. Sends one API call per case
(all populated fields batched into a single prompt) and returns
a data frame with field, parser_value, verdict
(\code{"agree"} / \code{"disagree"} / \code{"unclear"}), and a
one-sentence reason. Not authoritative -- the cached HTML is
the ground truth -- but a fast way to triage which rows a human
should re-read against the report.
Section-text terminator fix (parser correctness). The
section_text() helper used to stop only at the next <h2>,
so the LAST <h2 id="section_N"> block on a page (typically
section_8 -- analysis / decision) silently captured everything
to end-of-document, including the site's left-nav and footer.
This leaked phrases like "First Nations, Inuit and Métis
Liaison Program" and Twitter follow links into every report's
narrative_summary, supplemental_materials, and
mental_health_or_race_indications -- the latter would have
tagged every case in Ontario as "First Nation" regardless of
the report's actual content. The terminator now also stops at
<footer, <aside, <nav, whichever comes first after the
section anchor.
mental_health_or_race_indications expansion. Search scope
now includes section_5 (Affected Person), which is where many
reports state race / mental-health context. Keyword set
expanded with suicidal, psychotic, self-harm,
self harm, emotionally disturbed, EDP, Mental Health Act,
Inuit (alongside the existing Black / Indigenous / First
Nation / mental health / in crisis / racializ / racial set).
Shipped DRID manifest. inst/extdata/siu_drid_manifest.csv.gz
(~46 KB) ships with the package, listing 6,000 verified drids
(4,443 with parsed case_number, covering 2,218 unique cases as
of 2026-05-20). The harvester reads this floor automatically
via morie_fetch_siu() -- new cases above the manifest's max
are still discovered live.
html_to_text segfault fix. The previous C++ HTML tag stripper
used three std::regex_replace calls with .*? patterns; on at
least one drid in the 1..6000 sweep these recursed through the C
stack and aborted R with "segfault from C stack overflow",
killing the manifest build mid-run. Replaced with a linear single-
pass state machine (no recursion, no backtracking risk) plus a
defensive 4 MB input cap.
Rate-limited multi-fetch. .siu_http_get_many() now drives a
token-bucket throttle (default 4 req/s across the whole pool) with
exponential backoff on HTTP 429/502/503/504. morie_fetch_siu()
defaults to concurrency = 4L, rate_rps = 4.0. The previous
concurrency = 16-24 default was high enough to trigger WAF
interstitials on some networks (most visibly on GitHub Actions
Azure egress IPs), which returned short non-report HTML that
looked like data but wasn't.
morie_siu_refresh_manifest() — sweeps director's-report ids
1..max_drid (default 6000), records each id's HTTP status, body
size, and parsed case number, and writes a gzipped CSV manifest of
known-valid drids. The shipped manifest at
inst/extdata/siu_drid_manifest.csv.gz lets morie_fetch_siu()
short-circuit the ~30-50% of drids that have no published report,
saving bandwidth and reducing WAF-trigger risk.
Live max discovery, always. The harvester now always probes
past the live SIU index max (+300 drid margin, up from +150),
so reports added after the manifest snapshot are still captured.
The manifest is a floor on the known-valid id space, never a
ceiling on what's swept.
.siu_http_get_many_with_status() — new internal export
returning body + http_code + attempts in parallel slots, used by
the manifest builder and available for diagnostic scripts.
New: a generic open-data access layer, and a much wider dataset catalog.
morie_fetch() — a universal URL fetcher. It auto-detects the
resource format from the HTTP Content-Type header (falling back to
the URL extension) and parses CSV, TSV, JSON, XML, HTML, XLSX, and
ZIP-integrated files. Every step is overridable: pass an explicit
format, extra query params, or a zip_member to extract.morie_ckan_search() — discover datasets on any CKAN open-data
portal (open.canada.ca, data.ontario.ca, open.toronto.ca, or a
custom base URL). Returns one row per resource, with the
resource_id to feed into morie_fetch_ckan().morie_fetch_arcgis() — query any ArcGIS FeatureServer /
MapServer layer, paginating through the server transfer limit.morie_dataset_catalog() gains download_url,
zip_member, and arcgis_url columns and a six-tier
morie_load_dataset() resolver. CKAN resource ids were added for the
CCS 2018-2022/2023/2024 and CSUS 2023 PUMFs; direct-download URLs for
23 further datasets (CIHI indicator tables, StatCan and
Health-Infobase zip capsules); and verified ArcGIS layer URLs for the
three TPS crime series.morie_load_dataset(refresh = TRUE) — bypass the built-in
database and user cache to re-fetch a dataset from its remote
source, picking up time-to-time updates.Fix: Toronto Police Service open-data ingestion correctness and reliability.
tpshomicides and tpsshootings
entries in dataset_catalog.R advertised a 2014-present date
range. The Public Safety Data Portal publishes the Homicides and
Shootings & Firearm Discharges series from 2004; the catalog
metadata is corrected to 2004-present.morie_fetch_tps() pagination — the ArcGIS paging loop stopped
as soon as a page returned fewer rows than the requested page size.
A layer whose server-side maxRecordCount is below that size
returns short pages on every call, so the download was silently
truncated to the first page. The loop now pages on the server's
exceededTransferLimit flag, and a failed request aborts with an
error instead of caching a partial download.OCC_DATE is converted to UTC
by the ArcGIS platform; daily-resolution Hawkes fits now build the
date from the local-time OCC_YEAR/OCC_MONTH/OCC_DAY integer
fields so events near local midnight are binned to the correct day.Fix: CRAN source-package compliance for the vendored C++ core header.
src/ header extension — the R package vendors a copy of the
shared C++ numeric core. R CMD check --as-cran does not recognise
.hpp as a src/ file extension and emitted a WARNING, which
blocks CRAN submission. The vendored copy was renamed
morie_core.hpp to morie_core.h and the #include in
morie_fast.cpp updated to match. No behaviour change; the
canonical libmorie/morie_core.hpp is unchanged.Fix: complete the Docker image build fix; atomic release pipeline.
LICENSE into the
build stage, which scikit-build-core requires (license-files).
The builder now copies it; the image build is verified.Fix: the Docker container build for the v0.9.1 C/C++ core.
CMakeLists.txt and libmorie/ sources.New: a shared C/C++ computational backend and a Hawkes-process engine.
libmorie), bound into the R package via Rcpp. The same core
serves the Python and R sides.New: dataset availability auditing, more open-data sources, and in-place self-update.
check_datasets() dataset auditor — probes every entry in the
dataset catalogue and reports which datasets are reachable and which
need attention, classified by tier.morie.ingest.statcan adds the
Canadian Community Health Survey 2022 PUMF (StatCan 82M0013X) as the
cchs22 dataset, fetched on demand from the StatCan product page.morie.ingest.cihi adds five Canadian Institute
for Health Information indicator data tables (hospital stays for harm
caused by substance and alcohol use; youth integrated-youth-services
access), fetched on demand from cihi.ca.import morie performs a fail-silent,
daily-cached check against PyPI and prints a one-line notice when a
newer release exists. Opt out with MORIE_NO_UPDATE_CHECK.
(Python interface.)morie update command — checks PyPI and, with confirmation,
upgrades morie in place. (Python interface.)morie_load_cpads example is now wrapped in
\dontrun{}, so R CMD check --as-cran no longer errors on the
offline check farm.~/.cache/morie, or
$XDG_CACHE_HOME). A stale path calculation previously placed them
outside any writable location; MORIE_CACHE_DB still overrides.
Fixed identically on the R side, so the shared cache works.morie doctor --fix — the diagnostics command can now remediate
failed checks: install missing Python packages, create the cache
directory, and warn when a newer release is available. Plain
morie doctor stays diagnostic-only. (Python interface.)load_dataset() and check_datasets() now explain where it
comes from — the CKAN portal, an on-demand fetcher, or the local
path to place the file — via the new dataset_recommendation()
helper.New: the fairness & disparity-audit subsystem (morie.fairness).
A subsystem for auditing risk-assessment, recidivism, and predictive-policing systems for racial and other group disparities. morie does not deploy such systems — it measures whether an existing one encodes disparate treatment, so researchers and oversight bodies can hold those systems accountable.
predpol_calibration_audit
ranks areas by predicted risk against realised outcomes and tests
whether the disagreement tracks area demographics; paired with
predpol_score_disparity and a city-agnostic CityProfile layer so
the audit runs for any city. Python and R.predpol_temporal_audit computes the
four disparity metrics per (city, period) cell and surfaces temporal
instability and cross-city divergence. Python and R.morie[sim]
extra; JAX, not PyTorch, to stay lean).The methods are implemented from published descriptions — IBM AIF360; the SciencesPo Predictive-policing-Chicago project; Barman & Barman (arXiv:2603.18987); and the COMPAS audit in pbiecek's XAI Stories.
Security patch.
siu_fetch). The index-page link
parser used a repeated sub-pattern with \s* on both ends, which
could cause catastrophic (exponential) backtracking on a maliciously
crafted HTML page. The pattern is now linear-time; parsing of valid
SIU index pages is unchanged. (CodeQL py/redos, high severity.)User-Agent strings across the data-ingestion modules were stale
(morie/0.9.5.6–morie/0.6.1) and are now aligned to the release
version.License change. morie is now licensed under the GNU Affero General
Public License v3 or later (AGPL-3), on both the Python and R
sides.
moirais alias package has been removed.GPL-2.0-only as before.Documentation-only patch on top of 0.7.1. Supersedes the in-queue 0.7.1 submission for the rOpenSci pre-submission inquiry / next CRAN bump.
@examples coverage on exported functions: 100% (377/377). Up
from 19.9% in 0.7.1. ~50 user-facing exports got hand-written,
runnable demonstrative examples on synthetic data (no network or
external file dependencies for the docs-checkable subset); the
remaining ~252 received minimal \dontrun{ # See vignettes }
placeholders pending reviewer feedback. This was the primary
rOpenSci-readiness gap on 0.7.1.R CMD check --as-cran:
mrm_latin_square example now converts mrm_random_latin()'s
integer codes to letters before matching against LETTERS,
avoiding an all-NA outcome that crashed aov() with the
"contrasts can be applied only to factors with 2 or more levels"
error.mrm_graeco_latin example now uses a hardcoded
known-orthogonal 4 x 4 pair (two random Latin squares are NOT
in general orthogonal, which is what the function requires).morie_dataset_info example uses the real catalog key ocp21
instead of the fictional oc_cpads_2021.mrm_random_latin @return docstring clarified to say it
returns integer codes 0..k-1, not letters.morie_load_cpads.Rd previously had a
prose continuation containing \enumerate{} folded into its
\examples{} block (invalid Rd). Source rearranged so the
prose stays in \description{}.mrm-dataset-fetchers, mrm-empirical-callables,
and mrm-otis-walkthrough had their inst/doc/*.html outputs
rebuilt after the OTIS-expansion + MRM-acronym fixes from 0.7.1.AGPL-3.0-or-later in 0.7.3 — see that
entry.) The Linux-kernel adjuncts in kernel-module/ and daemon/
remain GPL-2.0-only (kernel ABI requirement) and are not part of
the CRAN tarball.ac (alert complexity)
and vm (volatility measure of placements, "regional-transition
count" alongside) are now the canonical operational terms.morie_mean,
morie_var, morie_cor_pearson, morie_normal_pdf,
morie_fast_available.Author compatibility: DESCRIPTION now carries
an explicit Author: field alongside the modern Authors@R: so
R CMD check passes on the 4.6.0 series.morie.mrm_classify_mandela() as the dual of the R-side
rmorie::mrm_classify_mandela() (which had shipped in v0.1.14).
All 25 v0.2.0-era callables now exist on both language sides.VERSION_INVENTORY.csv — every file that carries a version
string, its category (CURRENT vs HISTORICAL), and the
exact match.DEPENDENCIES.csv — every Python and R dependency with
name, version pin, license, and GPL-2.0-only compatibility.mrm_otis_* (5 fns, OTIS),
mrm_tps_* (4 fns, TPS), mrm_siu_* (3 fns, SIU), plus
mrm_tps_kulldorff_scan (space-time scan with MC permutations).
All have R + Python parity.fetch_tps_category (ArcGIS REST) and
fetch_siu_cases (on-demand scraper for the Ontario SIU public
Director's Reports). OTIS CKAN resource IDs registered for
a01/b01/b09/c11; loadable via morie_load_dataset().inst/extdata/ (random
1000-row b01 + b09 + c11 + tps_assault, ~420 KB total) so the
examples run offline.simulate_longitudinal_panel() — VAR(L) panel
simulator with structured covariance kernels.check_plugin_license() runtime guard, optional
out-of-tree kernel module (kernel-module/morie.c), optional
userspace audit daemon (daemon/morie_lsm.py).python -m morie.demo showcases every
new callable end-to-end on the integrated samples with rich-based
spinners + progress bars (DoubleML / Optuna style).results/).
Corrections shipped: Hill α 1.62 → 2.08; SDB 22% → 57%; Hawkes
Gamma → Weibull (hawkes-paper abstract typo); KM TTR 210 days
→ flagged as ID-misreading artefact (actual SIU TTR is 120 days);
LISA Assault 2024 quadrants 47/5/4/44 → verified 19/13/17/52.GPL-2.0-only SPDX (matching
the Linux kernel convention) across CITATION.cff, pyproject.toml,
both DESCRIPTION files, LICENSING.md, README, kernel module.python -m sphinx rebuilds with cleaner intro
prose for the API reference pages.morie_did(), morie_iv_2sls(), morie_rdd() -- unified
quasi-experimental entry points routing to the native toolboxes:
auto-staggered CS dispatch with a Goodman-Bacon contamination
warning, a Staiger-Stock weak-instrument refusal gate with the
Anderson-Rubin set, and RDD with the manipulation test and placebo
cutoffs bundled.morie_weight_* -- the full propensity-weighting family
(logistic/entropy/CBPS/overlap/stabilized/trimming/SuperLearner
stack/diagnostics) as a composable morie_weight class; weights
match WeightIt to 1e-8 and CBPS balance moments to < 1e-6.morie_did_sun_abraham(), morie_did_borusyak(),
morie_did_did2s() -- the three modern staggered-DiD estimators on
the native TWFE machinery; within 0.02 of the did2s package.morie_crim_etas(), morie_crim_hawkes_multivariate(),
morie_crim_near_repeat(), morie_crim_risk_terrain() -- the
criminology methods absent from the toolbox after the variant audit.inst/extdata/quasiex/ with
literature-replication tests, plus the phase-17 composition test
(DAG -> identify -> match -> weight -> estimate -> refute ->
report on real data).tests/cross/ against its reference package and
benchmarked in inst/benchmarks/:
matching (7 methods; modules 1-7), design-based IPW/svyglm to 1e-6
(8-9), double machine learning 40-60x faster than DoubleML (10),
R-learner causal forest 1.5-2.9x faster than grf (11), all four
meta-learners (12), the causal DAG toolkit with Bayes-Ball
identification and DoWhy-style refutation (13), the complete DiD
family — TWFE and event studies reproducing fixest to 1e-10,
Callaway-Sant'Anna with DRDID-identical influence functions,
Goodman-Bacon, DID-M, feTR weights, honest (Rambachan-Roth style)
sensitivity (14), Abadie synthetic control + synthetic DiD (15),
RDD with a native Imbens-Kalyanaraman bandwidth and McCrary test
(16), k-class IV (2SLS/LIML), two-step/CUE GMM, and NEW
morie_its() interrupted time series with native Newey-West (17),
NEW IRT: morie_irt_2pl(), morie_irt_grm(), morie_irt_eap()
(18), geostatistics: variogram ML + ordinary kriging equal to gstat
to 1e-6 (19), DSP: Butterworth/filtfilt/FIR/Savitzky-Golay/Welch/
DWT (20), Hawkes MLE (21), standalone SHA-256/HMAC/PBKDF2 pinned by
NIST/RFC vectors (22), native JSON/XML/HTML/Parquet parsers with
optional accelerators (23), and the MRM flagship:
morie_mrm_load_si_dataset(), morie_mrm_reconcile(),
morie_mrm_estimate_causal_effect(), morie_mrm_report() (24).morie_crypto_pqc_inventory().morie_safe_recode(), morie_safe_factor(),
morie_audit_categories(), morie_crosstab_verify(), and a
binary-treatment guard wired into the MRM pipeline, making the
silent category-mapping corruption class (numeric codes imported
without labels, positional recodes, reference releveling)
structurally impossible.morie_did_group_time_att
and morie_did_doubly_robust gain an se_convention argument
("reference" reproduces did/DRDID exactly; "bessel" keeps Bessel's
correction). morie_did_twoway_fe_weights supports type = "feTR"
only (the canonical diagnostic).spdep::moran.test(randomisation = FALSE).inst/CITATION at runtime.tests/cross/.cli_main() — single R-side dispatcher for the rmorie-cli analyze <subject> verb (otis/siu/nypd/cpd run real analyses; tps returns a guided
note). Forwards CLI flags as JSON with formal-intersection.morie_cpd_all_analyses() and morie_nypd_all_analyses() — new
single-call analysis suites for the Chicago/NYC policing subjects (crimes
by type, predpol area concentration, temporal, and race disparate-impact),
mirroring morie_otis_all_analyses. Offline on bundled fixtures.Non-breaking trust-boundary guards on the process-execution and
deserialization sinks. Defaults are unchanged; these add a kill-switch
and a single choke point (see the new internal R/aaa_exec_guard.R).
MORIE_NO_EXEC=1 kill-switch — now honoured by every sink that
spawns an external process: morie_agent(), morie_ask_percy()
(Python), morie_vertex_access_token() (gcloud), the morie_bricklayer()
pip install, and the DMT_Imaging git clone. Set it on shared/CI/locked
machines to disable all process execution.readRDS() sites that take a
user-supplied path (morie_cache_file(), morie_inspect_output(),
morie_ml_load()) now route through .morie_safe_readRDS(), which also
honours MORIE_NO_EXEC (an .rds file can execute code while loading).
First-party cache/bundled loads are unchanged.morie_entheo_clone_dmt_imaging() now rejects
branch names outside ^[A-Za-z0-9][A-Za-z0-9._/-]*$, closing an
option-injection vector (e.g. --upload-pack=...) into git clone.All system2() sinks continue to use argv lists (no shell-string
interpolation); that invariant is now documented in the guard file.
MAKEFLAGS=-j1), matching r-cmd-check/r-coverage: the ECOSolveR
SuiteSparse/AMD vendored Makefile races under parallel make on every
cache miss, which killed two consecutive main-branch pkgdown runs as
silent 80-140 minute runner hangs (2026-07-09).timeout-minutes: 120 so a wedged runner
fails fast instead of holding the run open for hours.morie_tps_* statphysics
functions previously accepted save_fig = TRUE and reported a
written figure while writing nothing (there was no png() call in
the file). Each now takes fig_dir = and writes a real PNG
(SDB attractiveness/density fields, Levy CCDF + Hill fit,
urban-scaling log-log fit, Lotka-Volterra trajectories, Turing
lattice, inspection-game phase heatmap, premise co-occurrence
network) -- or says plainly that it skipped.morie_tps_load_tps_dataset() and
morie_tps_load_tps() replace the never-shipped stubs the
statphysics suite guarded on ("NotYetPorted"): local project export
when present, otherwise a live fetch from the public TPS ArcGIS
portal. The data-seeded statphysics analyses now run anywhere.morie_tps_figures() gains "langevin" and "fokker_planck"
families (OU path fan + evolved-vs-stationary density).morie_otis_figures(): five OTIS exploratory figures
(placement-length survival tail with the 15-day Mandela threshold,
alert-prevalence trends, alert co-occurrence Cramer's V,
age-by-year, regional Pareto) rendered offline from the bundled
b01 sample.morie_tps_figures(): portable batch export of the TPS
stochastic diagnostic figures (Hawkes fit panel with time-rescaling
residuals + KS diagnostic; SARIMA hold-out panel), fetching live
from the public TPS ArcGIS portal and writing only under the
caller's out_dir. OTIS/SIU figure exports follow in the next
patch release..tps_stoch_date_series() now parses TPS month-name OCC_MONTH
and ArcGIS epoch-millisecond OCC_DATE values (previously every
live-fetched tps_stochastic analysis errored).morie_tps_hawkes_temporal_fit() deterministically spreads
same-day tied events within the day, fixing the degenerate
omega -> Inf decay on daily-resolution sources.cran-comments.md, added the
CRAN version/downloads badges and the hex logo (man/figures/logo.png,
generated reproducibly by data-raw/hex_logo.R).FutureLaunchError and the R-devel connection-diagnostic crash
introduced by {future} 1.70.0.future declared in Suggests (fixes the '::' import not declared
NOTE).morie_tox_*), cluster-robust double
machine learning (morie_dml_clustered), and spatiotemporal Hawkes
(morie_hawkes_st_*).morie_bricklayer() assembler + morie_core.h sync-check..otis_cluster_se(): use sum(grp^2, na.rm = TRUE). A factor cluster carrying levels absent from a subset (e.g. per-year analysis of an individual-clustered panel) made tapply() emit NA for the empty clusters, yielding an NA SE that errored downstream (if (se > 0): "missing value where TRUE/FALSE needed"). Empty clusters contribute nothing, so they are now dropped. Fixes morie_otis_irm_dml() on per-year clustered runs. Regression test added.README.md: removed the "(DOIs will be re-added once we re-deposit on Zenodo.)" promise. The Zenodo deposits for the morie publication set were taken down; we are not committing to re-depositing.inst/CITATION: removed the stale "also the R package source on Zenodo" comment.NEWS.md: removed Zenodo DOI references from the historical 0.9.5.7 changelog entry.morie_anchors_analyze (Phase 1.l) -- upstream anchors was archived from CRAN on 2022-03-06 (check problems not corrected). pak resolver could not solve the dependency, blocking CI.morie_causal_mediation (Phase 1.h) -- upstream causalweight was archived from CRAN on 2026-05-18 because its dependency LARF was archived. Same pak resolver failure.R/extenders_nonparam.R with 6 wrappers (morie_locfdr_estimate, morie_fdr_qvalues, morie_quantile_reg, morie_np_kernel_reg, morie_dp_gaussian_mixture, morie_lcmm_latent_class)R/extenders_spatial.R with 9 wrappers covering variograms, kriging, copulas, kernel PCA, spectral clustering, meta-analysis, multivariate normal sampling and CDF (morie_geostat_variogram, morie_geostat_krige, morie_copula_fit, morie_copula_sample, morie_kernel_pca, morie_spectral_cluster, morie_meta_rma, morie_mvnorm_sample, morie_mvnorm_pmv)R/extenders_rdd.R with 5 wrappers (morie_rdd_density_test, morie_rdd_local_randinf, morie_rdd_power_calc, morie_anchors_analyze, morie_anominate_ideal_points)morie_rdd_power_calc because morie_rdd_power already exists in R/rdd.R as a closed-form analytical power formula taking (n, tau, sigma); the simulation-based rdpower::rdpower surface is preserved alongside.A new file R/extenders_stats.R adds 17 wrapper-as-extender entry
points under the canonical morie_<pkg>_* prefix that delegate to
five CRAN statistics packages. Each function follows the
requireNamespace-guarded hard-error pattern used by the other
1.g/1.h/1.i/1.j extenders and returns a thin two-slot list with
$method (qualified upstream name) and $raw (upstream object):
morie_desc_cramers_v, morie_desc_kappa,
morie_desc_winsorize, morie_desc_gini, morie_desc_atkinson
-> DescTools::{CramerV, CohenKappa/KappaM, Winsorize, Gini, Atkinson}.morie_performance_check_model, morie_performance_r2,
morie_performance_check_collinearity,
morie_performance_check_outliers -> performance::{check_model, r2, check_collinearity, check_outliers}.morie_ppcor_partial, morie_ppcor_semipartial ->
ppcor::{pcor, pcor.test, spcor, spcor.test} (matrix-wise when
y/z are omitted, single-triple test otherwise).morie_coin_independence, morie_coin_wilcoxon,
morie_coin_oneway -> coin::{independence_test, wilcox_test, oneway_test}.morie_randtests_runs, morie_randtests_turning_point,
morie_randtests_bartels -> randtests::{runs.test, turning.point.test, bartels.rank.test}.DESCRIPTION: adds DescTools, ppcor, randtests to Suggests
(alphabetised). coin and performance were already listed.
Tests: tests/testthat/test-extenders-stats.R covers one happy
path per function, each gated by skip_if_not_installed().
Two new wrapper-as-extender entry points have been added to
R/did.R to close the Phase 1.e gap. Both follow the
requireNamespace-guarded hard-error pattern of the existing DiD
wrappers and ship under the canonical morie_did_* namespace:
morie_did_twoway_fe_weights(panel, group, time, treatment, outcome, type = "feTR", ...) -- thin interface to
TwoWayFEWeights::twowayfeweights (de Chaisemartin &
D'Haultfoeuille, 2020). Returns an
morie_did_twfe_diagnostics S3 list with the negative-weight
count, the sum of weights, and the full twowayfeweights object
as $raw. Complements morie_did_panel_fe (which estimates the
TWFE coefficient itself) and
morie_did_chaisemartin_dhaultfoeuille (which uses the same
identification argument to deliver the DID-M estimator).morie_did_synthdid_estimate(panel, unit, time, treatment, outcome, vcov_method = "placebo", ...) -- thin interface to
synthdid::synthdid_estimate (Arkhangelsky et al., 2021) under
the synthdid-canonical name. Parallel to the existing
morie_did_synthetic, which keeps the rmorie result-list shape;
this extender surfaces the full synthdid object and the requested
variance estimator (placebo, bootstrap, jackknife).DESCRIPTION: adds TwoWayFEWeights to Suggests. synthdid was
already listed in 0.9.5.12.
Tests: tests/testthat/test-did-extender.R covers both the happy
path (skipped when the optional package is not installed) and the
missing-package error path.
The sensitivity subsystem (R/sensitivity.R, ~733 LOC) keeps its
existing inline math as a fallback arm but cross-references the
canonical CRAN packages (rbounds, tipr, sensemakr, specr,
episensr) in the Rd files, and four new wrapper-as-extender entry
points have been added so MRM / paper callers can reach the full
surface of these packages from inside rmorie:
morie_sensitivity_evalue(estimate, se, sd, type = "OLS", ...) --
thin interface to the EValue::evalues.* dispatch family
(evalues.OLS, evalues.RR, evalues.OR, evalues.HR,
evalues.MD). Pairs with the typed e_value_rr/_or/_hr/_d
wrappers, which call the same backend but with a fixed scale.morie_sensitivity_tipping_point(estimate, smd, r2, ...) --
thin interface to tipr::tip for unmeasured-confounder tipping
points. Pairs with tipping_point_analysis (which targets
missing-data sensitivity rather than unmeasured confounders).morie_sensitivity_omitted_var_bias(model, treatment, benchmark_covariates, ...) -- thin interface to
sensemakr::sensemakr on a fitted lm. Pairs with
omitted_variable_bias (closed-form Cinelli-Hazlett robustness
value when only estimate + se + dof are available).morie_sensitivity_konfound(estimate, se, n, n_covariates, ...)
-- thin interface to konfound::pkonfound for the Frank et al.
(2013) percent-bias-to-invalidate and impact-threshold-of-a-
confounding-variable (ITCV).All four hard-error with a clear install.packages(...) message if
the optional dependency is missing, matching the Phase 1.e / 1.f
pattern.
rosenbaum_bounds, tipping_point_analysis, omitted_variable_bias,
specification_curve, and probabilistic_bias_analysis are kept
in-house (the rmorie result shapes are part of the public API and
the inline math remains the reference fallback). manski_bounds,
bias_adjusted_estimate, and sensitivity_summary are novel /
aggregator code with no clean CRAN counterpart.
DESCRIPTION: adds tipr, sensemakr, konfound to Suggests.
Tests: tests/testthat/test-sensitivity.R extended with eight new
test_that() blocks (one happy path + one missing-package error path
per new extender, all gated by skip_if_not_installed).
The causal-inference subsystem (~876 LOC) has been thin-wrapped over the canonical CRAN causal-inference packages while preserving every inline math fallback for CRAN-only installs (dual-arm pattern).
morie_estimate_propensity_scores() delegates to
WeightIt::weightit(method = "glm", estimand = "ATE") when
available; falls back to stats::glm(family = binomial()).morie_estimate_ate() / morie_estimate_att() /
morie_estimate_atc() inherit the WeightIt delegation through
the propensity-score helper; Hajek difference and
influence-function SE remain inline to preserve the closed-form
result shape.morie_estimate_aipw() keeps the AIPW score inline; the propensity
arm picks up WeightIt automatically. Rd cross-references AIPW::AIPW
as the canonical SuperLearner-based alternative.morie_estimate_g_computation() delegates the standardisation step
to stdReg::stdGlm() when available; falls back to the inline
glm() + treatment-flipped counterfactual contrast.morie_estimate_late() already used ivreg::ivreg(); now also
recognises AER::ivreg() as a second delegation arm, falling back
to manual 2SLS otherwise.morie_estimate_double_ml() / morie_estimate_irm() continue to
delegate to DoubleML::DoubleMLPLR / DoubleMLIRM when available;
fall back to the cross-fit ridge implementation.morie_e_value() delegates to EValue::evalue() when available;
falls back to the inline closed-form RR -> E-value formula.morie_sensitivity_rosenbaum() delegates to rbounds::psens()
when available; falls back to the inline sign-score bound.Four new extender functions are introduced for CRAN dependencies
that previously had no morie_* entry point:
morie_causal_impact(data, pre_period, post_period, model_args) ->
CausalImpact::CausalImpact() (Brodersen et al. 2015 Bayesian
structural time-series intervention analysis).morie_causal_weighting(data, treatment, covariates, method, estimand, ...) -> WeightIt::weightit() (full method palette:
glm / cbps / ebal / ps / energy / optweight).morie_causal_robust_se(model, type, cluster, ...) ->
sandwich::vcovHC() / vcovHAC() / vcovCL() (HC0-HC5, HAC,
one-way cluster-robust variance).morie_causal_mediation(y, d, m, x, trim, boot, ...) ->
causalweight::medweight() (semiparametric IPW direct / indirect
effect decomposition).The four extenders hard-error on missing packages (no inline
fallback) since each upstream implementation is too large to
re-implement compactly. New tests live in
tests/testthat/test-causal-extenders.R and are guarded with
skip_if_not_installed().
DESCRIPTION Suggests now lists AIPW, CausalImpact,
causalweight, rbounds, sensitivitymv, and stdReg so the
extenders and the delegation arms can find their upstream packages
when available.
The bootstrap / resampling subsystem (~867 LOC) has been re-routed
through the canonical CRAN packages while preserving the rmorie API
and the morie_bootstrap_result / morie_jackknife_result /
morie_permutation_test_result / morie_cv_result S3 return
shapes so that stat_commands, the print methods, and MRM
analyses keep working unchanged.
Thin-wrapped (with requireNamespace-guarded delegation arm + inline
fallback so the wrapper keeps working on minimal installs):
bootstrap() -- nonparametric bootstrap now delegates to
boot::boot (stratification via strata=) and boot::boot.ci
for percentile / basic / normal / bca CIs. The studentized
CI path and the cluster resampling path remain inline because
boot::boot.ci(type = "stud") requires a precomputed variance
estimator and boot::boot does not expose
cluster-of-clusters resampling for the rmorie
statistic(data) -> scalar signature.parametric_bootstrap() -- delegates to
boot::boot(sim = "parametric") with an MLE / ran.gen pair built
from the requested distribution.block_bootstrap() -- moving / stationary blocks delegate to
boot::tsboot(sim = "fixed" / "geom"); the circular-block path
stays inline because boot::tsboot does not expose a circular sim
mode.jackknife() -- delegates to bootstrap::jackknife when
installed (rmorie reconstructs pseudovalues / influence values
around the returned jack.values).permutation_test() / paired_permutation_test() -- inline
shuffle loops retained (the rmorie API returns the full null
distribution which downstream MRM code consumes); coin's
oneway_test(distribution = "approximate") and
symmetry_test(distribution = "approximate") are cross-referenced
as the canonical CRAN equivalents.bootstrap_632() -- inline .632 / .632+ retained because the
rmorie API takes naked model_fn / score_fn callables;
ipred::errorest(estimator = "632plus") is cross-referenced.repeated_cv() / leave_one_out_cv() -- delegate to
rsample::vfold_cv for fold construction when no stratification
or grouping is requested; caret::trainControl and
rsample::loo_cv cross-referenced.Kept as in-house implementations (no clean CRAN drop-in for the rmorie API):
subsampling() -- Politis-Romano-Wolf rate scaling over a
user-supplied statistic(data); np::npsubsample is closest
but is kernel-specific.delete_d_jackknife() -- generalised jackknife with custom
max_subsets enumeration cap; resample::jackknife is
cross-referenced.wild_bootstrap() -- returns the full resampled coefficient
distribution; sandwich::vcovBS (variance only) and
fwildclusterboot::boottest (p-value only) cross-referenced.Added four new extender entry points (thin pass-through to the canonical packages):
morie_boot_run(data, statistic, R, strata, ...) -- direct
boot::boot bridge with rmorie-style statistic(data) -> scalar
adapter.morie_boot_basic_ci(boot_obj, type, conf) -- direct
boot::boot.ci bridge returning tidy
list(perc = c(lo, hi), bca = c(lo, hi), ...).morie_rsample_bootstraps(data, times, ...) -- direct
rsample::bootstraps bridge returning an rset.morie_simpleboot_two(x, y, statistic, R, ...) -- direct
simpleboot::two.boot bridge for two-sample bootstrap of a
scalar statistic.DESCRIPTION: adds boot, bootstrap, coin, ipred, resample,
rsample, simpleboot to Suggests (sandwich was already
present).
Function inventory is preserved: 11 prior exports kept
(bootstrap, parametric_bootstrap, wild_bootstrap,
block_bootstrap, jackknife, delete_d_jackknife,
permutation_test, paired_permutation_test, subsampling,
bootstrap_632, repeated_cv, leave_one_out_cv) plus 4 new
(morie_boot_run, morie_boot_basic_ci,
morie_rsample_bootstraps, morie_simpleboot_two). All 20 (15
prior + 5 new) tests/testthat/test-bootstrap_methods.R
test_that blocks pass (73 expectations, 2 conditional skips when
rsample or simpleboot is not installed).
The treatment-effect / marginal-effects module has been thin-wrapped
around its canonical CRAN extender packages and gained a new family
of morie_effects_* wrappers over Vincent Arel-Bundock's
marginaleffects API plus the emmeans and broom ecosystems.
Thin-wrapped (existing API preserved; requireNamespace-guarded
delegation arm + inline fallback so the wrapper keeps working on
CRAN-only installs):
estimate_ate() -- guarded HC3 SE via sandwich::vcovHC +
lmtest::coeftest; falls back to naive model SE with a warning.estimate_plr(), estimate_pliv() -- already DoubleML-aware;
retained as is with style cleanups (snake-case locals,
stats::qnorm / stats::pnorm qualification, brace style).estimate_ate_gcomputation() -- delegates to stdReg::stdGlm()
for the canonical regression-standardisation backend (analytic
SE on the contrast scale, no bootstrap needed); falls back to
the legacy inline bootstrap implementation when stdReg is
absent.sensitivity_rosenbaum() -- delegates to rbounds::psens() when
rbounds is installed and the matched-pair count is sufficient;
falls back to the inline normal-approximation Wilcoxon
signed-rank bounds otherwise.e_value() -- delegates to EValue::evalues.OLS() when both
EValue is installed AND the caller supplies an outcome
standard deviation via the new sd_y argument; falls back to
the closed-form continuous-scale RR proxy so the Python and R
ports stay numerically aligned by default.New extender wrappers (each a thin pass-through; the underlying package's native object is returned verbatim so downstream code keeps working with the canonical API):
morie_effects_emmeans(model, specs, ...) ->
emmeans::emmeans(). Use emmeans::pairs() /
emmeans::contrast() on the returned emmGrid for pairwise or
custom contrasts.morie_effects_predictions(model, newdata, ...) ->
marginaleffects::predictions().morie_effects_comparisons(model, variables, ...) ->
marginaleffects::comparisons().morie_effects_slopes(model, variables, ...) ->
marginaleffects::slopes().morie_effects_tidy(model, ...) -> broom::tidy() (with a
summary()-based fallback frame for the lm / glm classes
rmorie's MRM pipeline ships, so downstream tidy callers work
even on minimal-Suggests CI runs).The effects and margins packages are added to Suggests for
the cross-reference path -- users who want Fox's effect() /
predictorEffects() or Leeper's Stata-style margins::margins()
can call them directly on a model fitted via rmorie, without an
intervening wrapper.
DESCRIPTION: adds broom, effects, emmeans, marginaleffects,
margins, performance, rbounds, stdReg to Suggests.
Net: R/effects.R grows from 461 to ~640 LOC because the legacy
treatment-effect functions now carry both a CRAN-delegation arm
AND an inline fallback (CRAN policy does not allow Suggests to be
hard required), plus 5 new extender wrappers + 1 shared helper.
All 6 legacy exports preserved (estimate_ate, estimate_plr,
estimate_pliv, estimate_ate_gcomputation,
sensitivity_rosenbaum, e_value); 5 new exports added
(morie_effects_emmeans, morie_effects_predictions,
morie_effects_comparisons, morie_effects_slopes,
morie_effects_tidy).
The multiple-testing-correction subsystem (~916 LOC) has been
rewritten to forward to the canonical CRAN / Bioconductor packages
where one exists. Every wrapper preserves the rmorie API and the
morie_multiple_testing_result / morie_rich_result S3 shape so
that the stat_commands dispatcher, the
print.morie_multiple_testing_result method, and MRM analyses keep
working unchanged.
Thin-wrapped (with requireNamespace-guarded delegation arm + inline
fallback so the wrapper keeps working on CRAN-only installs):
bonferroni(), holm(), hochberg(), hommel(),
benjamini_hochberg(), benjamini_yekutieli() -- already
stats::p.adjust wrappers, retained as-is.sidak(), holm_sidak() -- inline closed-form math; mutoss
cross-referenced in Rd as the canonical step-down equivalent.storey_q() -- delegates to qvalue::qvalue (Bioconductor) when
installed; falls back to the inline Storey cutoff otherwise.estimate_pi0() -- delegates to qvalue::pi0est for storey /
bootstrap methods when installed; falls back to inline
estimators.fisher_combined() -- delegates to poolr::fisher when installed.stouffer_combined() -- delegates to poolr::stouffer when
installed and no weights are supplied (weighted Stouffer stays
inline because poolr does not take per-test weights).tippett_combined() -- delegates to poolr::tippett when
installed.harmonic_mean_p() -- delegates to harmonicmeanp::p.hmp when
installed.n_effective_tests() -- delegates to poolr::meff (Galwey /
Li-Ji / Nyholt) when installed.fixed_sequence(), fallback_procedure() -- gMCP cross-
referenced in Rd as the canonical graphical-MCP equivalent;
inline implementations retained to preserve the rmorie
stage-list return shape.Kept as in-house implementations (no clean CRAN drop-in for the rmorie API):
cauchy_combination() -- Liu and Xie 2020; ACAT is GitHub-only.hierarchical_bonferroni() -- rmorie-specific stage-list return
shape; gMCP covers the concept with a different graphical API.local_fdr() -- Efron empirical-Bayes KDE shape that locfdr
does not match in return structure.permutation_fwer(), permutation_fdr() -- step-down max-T and
empirical-null-p FDR over user-supplied null matrices; no CRAN
function exposes the same API.adjust_p_values() -- front-end dispatcher across the rmorie
wrappers; consumed by stat_commands.DESCRIPTION: adds poolr, qvalue, gMCP, harmonicmeanp,
multcomp, mutoss to Suggests.
Net: R/multiple_testing.R grows from 916 to 1118 LOC because each
thin-wrap function now carries both a CRAN-delegation arm AND an
inline fallback (CRAN policy does not allow Suggests to be hard
required, and qvalue is Bioconductor only). Function inventory
is unchanged: 25 exports preserved (bonferroni, sidak, holm,
hochberg, hommel, holm_sidak, benjamini_hochberg, bh,
benjamini_yekutieli, by_fdr, storey_q, fisher_combined,
stouffer_combined, tippett_combined, simes_combined,
harmonic_mean_p, cauchy_combination, fixed_sequence,
fallback_procedure, hierarchical_bonferroni, estimate_pi0,
adjust_p_values, n_effective_tests, local_fdr,
permutation_fwer, permutation_fdr, plus the
print.morie_multiple_testing_result S3 method). All 85
tests/testthat/test-multiple_testing.R assertions pass on a
fallback-only install.
The DiD subsystem (~1,719 LOC) has been rewritten to forward to the canonical CRAN packages instead of carrying ~700 LOC of base-R fallback code:
morie_did_panel_fe() is now a thin wrapper over
fixest::feols(y ~ d | unit + time) with cluster-robust SE and
hard-errors if fixest is not installed (the base-R two-way
within-transform fallback has been removed).morie_did_event_study() is a thin wrapper over fixest::feols
with fixest::i(rel_time, ref = -1) relative-time dummies plus
unit and time fixed effects; hard-errors if fixest is missing.morie_did_group_time_att() is a thin wrapper over did::att_gt
for the Callaway-Sant'Anna group-time ATTs and hard-errors if
did is not installed (the base-R bootstrap fallback has been
removed).morie_did_doubly_robust() is a thin wrapper over
DRDID::drdid_rc (Sant'Anna-Zhao 2020 repeated-cross-section
doubly-robust DiD) and hard-errors if DRDID is missing
(the hand-written GBM-or-logistic + linear-or-GBM bootstrap
fallback has been removed). The ps_model / or_model
arguments are retained for back-compat but ignored; DRDID uses
logistic propensity and linear outcome regression internally.morie_did_bacon_decomposition() is a thin wrapper over
bacondecomp::bacon and hard-errors if bacondecomp is not
installed (the base-R timing-pair enumeration fallback has been
removed).morie_did_chaisemartin_dhaultfoeuille() is a thin wrapper over
DIDmultiplegt::did_multiplegt and hard-errors if
DIDmultiplegt is not installed (the base-R switcher-comparison
bootstrap fallback has been removed).morie_did_synthetic() continues to delegate to
synthdid::synthdid_estimate (no change; synthdid is the only
R implementation).morie_did_sensitivity_analysis() keeps its delta-bound CI
sweep but the Rd now cross-references
HonestDiD::createSensitivityResults_relativeMagnitudes as the
reference implementation of Rambachan-Roth 2023 for event-study
estimates.The OLS-based wrappers (morie_did_2x2,
morie_did_repeated_cross_section, morie_did_triple_difference,
morie_did_continuous_treatment, morie_did_fuzzy) continue to
use the in-package .morie_did_ols_robust_se helper because the
specs are simple OLS / 2SLS regressions and a CRAN dependency for
trivially-short OLS would be a regression. The same helper is
reused by morie_did_wild_cluster_bootstrap, which remains
base-R by design (fwildclusterboot is GitHub-only; see 0.9.5.12
NEWS).
Aggregators that consume DiD output and produce rmorie-specific
tables (morie_did_aggregate_gt_att, morie_did_staggered,
morie_did_parallel_trends_data,
morie_did_test_parallel_trends, morie_did_placebo_test_*,
morie_did_heterogeneous, morie_did_diagnostics) are unchanged.
Net: R/did.R shrinks from 1,719 to 1,463 LOC (-256, -15%);
all 22 morie_did_* exports preserved; result-list shape is
unchanged so downstream callers see the same fields.
DESCRIPTION: adds DRDID, HonestDiD, DIDmultiplegt to
Suggests (fixest, did, bacondecomp, synthdid were
already listed).
The matching subsystem (~2,183 LOC) has been rewritten to forward to the canonical CRAN packages instead of carrying ~950 LOC of base-R fallback code:
morie_matching_nearest_neighbor(), morie_matching_exact(),
morie_matching_cem(), morie_matching_mahalanobis(),
morie_matching_optimal_pair(), morie_matching_full(),
morie_matching_subclassify(), morie_matching_variable_ratio()
are now thin wrappers over MatchIt::matchit() and now hard-error
if MatchIt (or its optional optmatch back end) is not installed.morie_matching_genetic() is a thin wrapper over
Matching::GenMatch() + Matching::Match() and hard-errors if
Matching is missing.morie_matching_entropy_balance() is a thin wrapper over
WeightIt::weightit(method = "ebal") (or ebal::ebalance as a
fallback) and hard-errors if neither is installed.morie_matching_balance(), morie_matching_balance_table(),
morie_matching_love_plot_data() are kept (they return the
morie_balance_result shape downstream MRM code consumes) but
the Rd cross-references now point users at cobalt::bal.tab() /
cobalt::love.plot() for richer balance reporting.morie_matching_cardinality() keeps its iterative-caliper
heuristic; the Rd now cross-references
designmatch::cardmatch() for the exact
mixed-integer-programming alternative.The carceral-domain helpers (morie_matching_att_matched /
ate_matched / atc_matched, morie_matching_abadie_imbens_se,
morie_matching_rosenbaum_bounds, morie_matching_doubly_robust,
morie_matching_multi_treatment, morie_matching_longitudinal,
morie_matching_quality, morie_matching_overlap,
morie_matching_estimate_propensity / _trim_propensity /
_common_support) are unchanged - they encode rmorie-specific output
shapes (morie_match_result, morie_te_result) that the MRM /
SIU / OTIS code paths depend on.
Net: R/matching.R shrinks from 2,183 to 1,586 LOC (-597, -28%);
all 27 morie_matching_* exports preserved; behaviour-compatible for
callers that already have MatchIt installed (which is the case for
all matching tests in the rmorie suite).
DESCRIPTION: adds cobalt, designmatch to Suggests.
To reduce code duplication with established CRAN packages and address
rOpenSci feedback on fn_call_network_size, the following functions
have been removed in favour of their well-maintained CRAN equivalents:
| Removed from rmorie | Use instead |
|---|---|
| cohens_d / morie_cohens_d | effectsize::cohens_d |
| cramers_v / morie_cramers_v | effectsize::cramers_v |
| eta_squared / morie_eta_squared | effectsize::eta_squared |
| hedges_g / morie_hedges_g | effectsize::hedges_g |
| morie_effective_sample_size | posterior::ess_basic or coda::effectiveSize |
| morie_find_project_root | here::here() or rprojroot::find_root() |
| fleiss_kappa | irr::kappam.fleiss |
| kruskal_wallis | stats::kruskal.test (base R) |
| shapiro_wilk | stats::shapiro.test (base R) |
| anderson_darling | nortest::ad.test |
| jarque_bera | tseries::jarque.bera.test |
Install the replacements with
install.packages(c("effectsize", "irr", "nortest", "tseries", "here")).
morie_two_sample_t_test() / morie_chi_square_test() /
morie_anova_one_way() now return list fields named cohens_d /
cramers_v / eta_squared (was morie_cohens_d / morie_cramers_v /
morie_eta_squared); the computation is inlined and unchanged.
Internal callers of morie_find_project_root() now go through a private
.morie_project_root() wrapper around here::here().
DESCRIPTION: adds here to Imports; adds effectsize, irr, tseries
to Suggests.
calculate_ebac(), is_over_legal_limit(),
calculate_ipw_weights(), estimate_irm() (DoubleML wrapper),
infer_measurement_level(), profile_dataset(), suggest_analysis_plan(),
compare_nested_logistic_models(), run_treatment_effects_analysis(),
run_weighted_logistic_analysis(), inspect_output(),
verify_statistical_output().morie_power_t_test,
morie_power_prop_test, sample_size_logistic), signal-processing primitives
(Butterworth filters, Higuchi fractal dimension, Hurst exponent),
dataset profiling, OTIS correctional-data analysis, and the MRM
(Multilevel Reconciliation Methodology) framework.morie
package on PyPI. Both expose the same conceptual public API; each uses
its native language's idioms and ML ecosystem (R: mlr3 + DoubleML;
Python: scikit-learn + DoubleML).estimate_irm() is a thin R wrapper around DoubleML::DoubleMLIRM from
the CRAN DoubleML package; DoubleML, mlr3, and mlr3learners are in
Suggests and the function gates them with requireNamespace().