This document describes the Magic Toybox Analytic Engine's public feeds, their field mapping to database schema, as well as its normalization and subsequent derivation of target analytics: TAM/SAM/SOM sizing and for building supply & demand models, including elasticity estimation.
The goal of this data model is to acquire these data points across the range of customer-facing (B2C) US-based businesses.
TAM (Total Addressable Market)
SAM (Serviceable Available Market)
SOM (Serviceable Obtainable Market)
Elasticity
Supply Curve
Demand Curve
Price Optimization
Census Bureau
Bureau of Labor Statistics
Federal Reserve Economics Data (FRED)
Bureau of Econmic Analysis
Securites & Exchange Commission
Census Bureau
The United States Economic Census is the U.S. federal government's official five-year measure of American business and the economy. It is conducted by the U.S. Census Bureau, and response is required by law. Forms go out to nearly 4 million businesses, including large, medium and small companies representing all U.S. locations and industries. Respondents are asked to provide a range of operational and performance data for their companies. Trade associations, chambers of commerce, and businesses use information from the economic census for economic development, business decisions, and strategic planning purposes. The last Economic Census was conducted for the year 2022.
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
| Report | ECN (all us) | ecn2022_us.txt | TXT | ecn2022_us.txt | stg.ecn_2022_us |
| Report | ECN (all states) | ecn2022_st.txt | TXT | ecn2022_st.txt | stg.ecn_2022_states |
| Report | ECN (all counties) | ecn2022_co.txt | TXT | ecn2022_co.txt | stg.ecn_2022_counties |
ECN Basic dat is subject to the following distortions that affect the interrelation between the us, state and county data sets, and will also likely distort its accuracy with respect to other reports measuring the same subject matter.
Suppression
Certain cells (industry × geography) are blanked to protect confidentiality when few establishments would be disclosed.
Location-based establishment counts (not registration)
Counts are tied to the physical location of establishments, not to where the parent firm is incorporated or registered.
Noise infusion / disclosure avoidance
Published values may be rounded or perturbed with statistical noise, so subtotals do not necessarily add up to higher-level totals.
Sampling / coverage (exclusion of nonemployers)
ECN Basic covers employer establishments only. Nonemployers (sole proprietors without payroll) are in the Nonemployer Statistics (NES) dataset.
Address and NAICS coding inconsistencies
Errors in geocoding or classification, or changes in NAICS vintages, can introduce discrepancies between local and national figures.
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
| Report | CBP (all us) | cbp23us.txt | TXT | cbp23us.txt | raw.cbp_2023_us |
| Report | CBP (all ???) | cbp23csa.txt | TXT | cbp23csa.txt | raw.cbp_2023_csa |
| Report | CBP (all ???) | cbp23msa.txt | TXT | cbp23msa.txt | raw.cbp_2023_msa |
| Report | CBP (by state) | cbp23st.txt | TXT | cbp23st.txt | raw.cbp_2023_st |
| Report | CBP (by county) | cbp23co.txt | TXT | cbp23co.txt | raw.cbp_2023_co |
| Report | CBP (by state) | cbp23pr_ia_st.txt | TXT | cbp23pr_ia_st.txt | stg.cbp_2023_ia_st_raw |
| Report | CBP (by county) | cbp23pr_ia_co.txt | TXT | cbp23pr_ia_co.txt | stg.cbp_2023_ia_co_raw |
| Report | CBP (all ???) | zbp23detail.txt | TXT | zbp23detail.txt | stg.cbp_2023_detail_raw |
| Report | CBP (all ???) | zbp23totals.txt | TXT | zbp23totals.txt | stg.cbp_2023_totals_raw |
ECN Basic dat is subject to the following distortions that affect the interrelation between the us, state and county data sets, and will also likely distort its accuracy with respect to other reports measuring the same subject matter.
Suppression
Employment and payroll cells may be withheld where few establishments exist, to protect confidentiality.
Location-based establishment counts
Counts reflect establishments’ operating addresses, not parent company registration.
Disclosure avoidance
Published values may include noise infusion or controlled rounding, so county totals do not exactly sum to state or national totals.
Coverage
CBP includes employer establishments only. Nonemployers are excluded and must be sourced from NES.
NAICS classification
Subject to industry coding changes over time; vintages differ across releases.
ECN Basic dat is subject to the following distortions that affect the interrelation between the us, state and county data sets, and will also likely distort its accuracy with respect to other reports measuring the same subject matter.
Scope
Covers businesses without paid employees (sole proprietors, partnerships, etc.) based primarily on IRS tax return data.
Location-based counts
Assigned by mailing or reported address, not necessarily where economic activity occurs.
Disclosure avoidance
Noise infusion and suppression methods apply, so local data may not sum to national totals.
Comparability
Cannot be directly added to CBP or ECN counts without adjustment, since employer and nonemployer definitions differ.
Classification
Industry coding (NAICS) follows reported information; some businesses may be miscoded or classified inconsistently.
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
| Variables | ACS (c profile) | acs5_cprofile_vars_2023.json | JSON | acs5_cprofile_vars_2023.json | raw.acs_cprofile |
| Variables | ACS (profile) | acs5_profile_vars_2023.json | JSON | acs5_profile_vars_2023.json | raw.acs_profile |
| Variables | ACS (subject) | acs5_subject_vars_2023.json | JSON | acs5_subject_vars_2023.json | raw.acs_subject |
| Variables | ACS (detailed) | acs5_detailed_vars_2023.json | JSON | acs5_detailed_vars_2023.json | raw.acs_detailed |
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
xxx
| Type | Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|---|
This list constitutes an open set of reference catalogs used in normalization of data labels across vintages and platforms
| Report | Link | Format | Saved file(s) | Intended table |
|---|---|---|---|---|
| NAICS Structure (2022) | 2022_NAICS_Structure.xlsx | XLSX | data/ref/naics/2022_NAICS_Structure.xlsx |
core.dim_naics |
| Gazetteer Counties (2023) | 2023_Gaz_counties_national.zip | ZIP → TXT | data/ref/gazetteer/2023_Gaz_counties_national.txt |
core.dim_geography |
| Gazetteer ZCTAs (2023) | 2023_Gaz_zcta_national.zip | ZIP → TXT | data/ref/gazetteer/2023_Gaz_zcta_national.txt |
core.dim_geography |
| Gazetteer CBSAs (2023) | 2023_Gaz_cbsa_national.zip | ZIP → TXT | data/ref/gazetteer/2023_Gaz_cbsa_national.txt |
core.dim_geography |
| Gazetteer CSAs (2023) | 2023_Gaz_csa_national.zip | ZIP → TXT | data/ref/gazetteer/2023_Gaz_csa_national.txt |
core.dim_geography |
| CBP Employment-size classes (EMPSZES, 2021) | API snapshot (JSON) | JSON | data/ref/empsz/2021_empsz.json |
core.dim_emp_size |
| ACS 5-year variables (2023) | acs/acs5/variables.json |
JSON → CSV | data/ref/acs/acs5_vars_2023.{json,csv} |
ref.ref_vars (dataset='acs5') |
| ACS 5-year Subject variables (2023) | acs/acs5/subject/variables.json |
JSON → CSV | data/ref/acs/acs5_subject_vars_2023.{json,csv} |
ref.ref_vars (dataset='acs5_subject') |
| ACS 5-year Profile variables (2023) | acs/acs5/profile/variables.json |
JSON → CSV | data/ref/acs/acs5_profile_vars_2023.{json,csv} |
ref.ref_vars (dataset='acs5_profile') |
| ACS 5-year Comparison Profile variables (2023) | acs/acs5/cprofile/variables.json |
JSON → CSV | data/ref/acs/acs5_cprofile_vars_2023.{json,csv} |
ref.ref_vars (dataset='acs5_cprofile') |
| Economic Census “ECN Basic” variables (2022) | ecnbasic/variables.json |
JSON → CSV | data/ref/ecn/ecnbasic_vars_2022.{json,csv} |
ref.ref_vars (dataset='ecnbasic') |
| NES Legal Form of Organization (2023) | nonemp?get=LFO,… |
JSON → CSV | data/ref/nes/nes_lfo_2023.json, data/ref/nes/lfo_2023.csv |
ref.ref_nes_lfo |
| NES Receipts size class (RCPSZFI, 2023) | nonemp?get=RCPSZFI,… |
JSON → CSV | data/ref/nes/nes_rcpszfi_2023.json, data/ref/nes/rcpszfi_2023.csv |
ref.ref_nes_rcpszfi |
| NES Years-in-business size (YIBSZFI, 2023) | nonemp?get=YIBSZFI,… |
JSON → CSV | data/ref/nes/nes_yibszfi_2023.json, data/ref/nes/yibszfi_2023.csv |
ref.ref_nes_yibszfi |
| NES Receipts-per-firm size (URSZFI, 2023) | nonemp?get=URSZFI,… |
JSON → CSV | data/ref/nes/nes_urszfi_2023.json, data/ref/nes/urszfi_2023.csv |
ref.ref_nes_urszfi |
| NAPCS Structure (2017) | 2017 NAPCS Structure XLSX | XLSX → CSV | data/ref/napcs/napcs_2017.{xlsx,csv} |
ref.dim_napcs (vintage='2017') |
| NAPCS Structure (2022) | 2022 NAPCS Structure XLSX | XLSX → CSV | data/ref/napcs/napcs_2022.{xlsx,csv} |
ref.dim_napcs (vintage='2022') |
| NAPCS Concordance 2022↔2017 | Concordance XLSX | XLSX → CSV | data/ref/napcs/napcs_2022_to_2017.{xlsx,csv} |
ref.ref_napcs_concordance_2022_2017 |