Module: OSINT/DataAcquisition

Dynamic Political Intelligence Statistics Loader

Core OSINT data acquisition module for the Riksdagsmonitor Intelligence Platform. Implements automated statistical intelligence collection from the CIA Platform (Citizen Intelligence Agency) production database exports.

Intelligence Methodology

This module implements systematic intelligence collection following OSINT best practices:

  • Source Authority: Direct database exports from CIA Platform (verified source)
  • Data Freshness: Nightly updates ensure 24-hour maximum staleness
  • Data Integrity: CSV parsing with validation and error handling
  • Fallback Strategy: Local-first with remote fallback for resilience

Data Source Intelligence

Primary Source: extraction_summary_report.csv (CIA Platform ETL report)

  • Columns: object_type, object_name, status, row_count, error_message, extraction_time
  • Scope: 349 MPs, 8 parties, 50+ years of parliamentary data
  • Update Frequency: Nightly automated ETL pipeline
  • Data Coverage: Votes, documents, committees, government proposals, risk violations

OSINT Collection Strategy

  1. Local-First Acquisition: Attempt local cached data (cia-data/)
  2. Remote Fallback: GitHub raw content as backup source
  3. Validation: Parse and verify data structure before use
  4. Error Resilience: Graceful degradation if sources unavailable

Intelligence Metrics Tracked

Hero Statistics (Homepage dashboard):

  • Historical political figures (person_data)
  • Total recorded votes (view_riksdagen_vote_data_ballot_politician_summary)
  • Parliamentary documents (document_data)
  • Rule violations detected (rule_violation)
  • Government proposals (view_riksdagen_goverment_proposals)
  • Committee decisions (view_riksdagen_committee_decisions)

Detailed Intelligence (Analysis sections):

  • Committee activities and proposals
  • Document lifecycle tracking
  • Party membership and voting blocs
  • Government role assignments
  • Ballot summaries and voting patterns

GDPR Compliance

Version:
  • 1.0.0
Since:
  • 2024
Author:
  • Hack23 AB - Intelligence Platform Team
License:
  • Apache-2.0
Source:
See:

Members

(inner, constant) LOCAL_CSV :string

Local cached OSINT data path for faster load times and offline resilience. Primary source in local-first acquisition strategy.

Type:
  • string
Source:

(inner, constant) REMOTE_CSV :string

Remote OSINT data source URL (CIA Platform GitHub repository) Used as fallback when local cached data is unavailable.

Type:
  • string
Source:

(inner, constant) STAT_MAPPINGS :Object.<string, string>

Intelligence metric mapping: DOM identifier → CIA database object name

Each entry connects a UI statistic element to its authoritative data source in the CIA Platform extraction report. This mapping ensures traceability from displayed intelligence to source data.

Type:
  • Object.<string, string>
Properties:
Name Type Description
stat-historical-persons string

Total political figures tracked
stat-total-votes string

Comprehensive voting record count
stat-total-documents string

Parliamentary document archive size
stat-rule-violations string

Detected anomalies and rule breaches
stat-government-proposals string

Executive branch legislative initiatives
stat-committee-decisions string

Committee-level decision tracking
Source:

Methods

(async, inner) fetchCSV() → {Promise.<(string|null)>}

Multi-source CSV fetch with intelligent fallback strategy. Implements resilient OSINT collection by attempting local cached data first, then falling back to remote GitHub source if local is unavailable.

Intelligence Collection Strategy

  1. Primary: Local cached data (cia-data/ directory)

    • Advantage: Sub-second response, offline capability
    • Source: Nightly automated data pipeline

  2. Fallback: Remote GitHub raw content

    • Advantage: Always available, continuously updated
    • Source: CIA Platform master branch

Validation Logic

  • HTTP 200 response required
  • Non-empty text content
  • Minimum 3 lines (header + 2 data rows) for validity
Source:
Throws:

Silently catches and logs fetch errors, continues to next source

Type
Error
Returns:

CSV text content or null if all sources fail

Type
Promise.<(string|null)>
Example
const csvData = await fetchCSV();
if (csvData) {
  const intelligence = parseCSV(csvData);
  updateDashboard(intelligence);
}

(async, inner) loadStats() → {Promise.<void>}

Master Intelligence Statistics Loader

Orchestrates the complete intelligence data acquisition, parsing, and display pipeline. This is the main entry point for populating the platform with real-time political intelligence metrics from the CIA Platform database.

Intelligence Pipeline

1. ACQUISITION  → fetchCSV() - Multi-source data retrieval
2. PARSING      → parseCSV() - CSV to structured objects
3. VALIDATION   → Filter by status === 'success'
4. MAPPING      → Apply STAT_MAPPINGS lookup
5. DISPLAY      → updateStat() for each metric

Data Validation

Only processes extraction records with:

  • status === 'success' (successful ETL extraction)
  • Non-empty object_name (valid data source identifier)
  • Non-empty row_count (actual metric value)

Intelligence Quality Control

  • Logs successful load with update count
  • Warns on fetch failures (graceful degradation)
  • Tracks coverage: updated stats / total mapped stats
Source:
Throws:

Catches and logs all errors, never crashes platform

Type
Error
Returns:

Completes when all intelligence metrics are updated

Type
Promise.<void>
Example
// Automatically called on DOMContentLoaded
await loadStats();
// Console: "✅ Stats loaded from extraction_summary_report.csv (24/24 stats updated)"

(inner) parseCSV(text) → {Array.<Object>}

Lightweight CSV parser for intelligence data transformation. Converts raw CSV text into structured intelligence objects for analysis.

Parsing Logic

  1. Split text into lines (newline-delimited)
  2. Extract header row (first line defines schema)
  3. Parse data rows (comma-separated values)
  4. Map values to header keys for object creation

Data Integrity

  • Handles empty cells gracefully
  • Trims whitespace from all values
  • Returns empty array if invalid input
Parameters:
Name Type Description
text string

Raw CSV text content
Source:
Returns:

Array of intelligence data objects with properties matching CSV headers

Type
Array.<Object>
Example
const csvText = "object_name,row_count,status\nperson_data,45678,success";
const data = parseCSV(csvText);
// Returns: [{ object_name: 'person_data', row_count: '45678', status: 'success' }]

(inner) updateStat(identifier, value) → {void}

Updates all DOM elements displaying a specific intelligence metric. Implements multi-element synchronization to ensure consistency across hero statistics, dashboard panels, and intelligence sections.

Update Strategy

  1. Number Formatting: Locale-aware formatting with thousands separators
    • Raw: 45678 → Display: "45,678"
  2. Multi-Element: Updates both ID-based and data-attribute selectors
    • <span id="stat-total-votes">...</span>
    • <span data-stat-id="stat-total-votes">...</span>
  3. Safe Updates: Validates value before DOM manipulation

Intelligence Display Principles

  • Readability: Human-friendly number formatting
  • Consistency: Same metric shown identically across UI
  • Accessibility: Text content updates for screen readers
Parameters:
Name Type Description
identifier string

Stat identifier (matches data-stat-id attribute or element ID)
value number | string

Intelligence metric value to display
Source:
Returns:
Type
void
Example
updateStat('stat-total-votes', 1234567);
// Updates all elements: <span id="stat-total-votes">1,234,567</span>
// And: <span data-stat-id="stat-total-votes">1,234,567</span>