Shannon Entropy Market Analysis

Summary

Shannon Entropy Market Analysis applies information theory to quantify trader behavior unpredictability and its correlation with market volatility. The project reveals counterintuitive patterns where predictable panic behavior (low entropy) correlates with high volatility, while diverse trading behavior (high entropy) correlates with market stability.

Key Findings

Counterintuitive Market Patterns:

Market Crashes: Low entropy (0.879 bits) + Very high volatility (6.555)
Normal Trading: High entropy (1.267 bits) + Moderate volatility (2.770)
Market Stress: Mixed entropy (1.113-1.147 bits) + High volatility (4.336-4.626)

Core Insight: When traders become predictable, markets become unpredictable. This challenges traditional assumptions about market efficiency.

Technical Implementation

Core Entropy Calculation:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
double shannon_entropy(std::vector<int> actions) {
    std::map<int, int> counts;
    int total = actions.size();
    for (int action : actions) {
        counts[action]++;
    }
    double entropy = 0.0;
    for (auto& pair : counts) {
        double p = (double)pair.second / total;
        if (p > 0) entropy -= p * std::log2(p);
    }
    return entropy;
}

Validation Results:

Unit Tests: 100% pass rate (exact entropy calculations)
Robustness Tests: 15/15 edge cases handled
Market Validation: 60 windows across 4 scenarios
Mathematical Accuracy: Achieves theoretical maximum entropy (1.585 bits)

Research Applications

Risk Management: Low entropy + high volatility = potential crash signal Market Timing: Entropy changes precede volatility spikes
Behavioral Analysis: Quantifies market sentiment complexity Trading Strategy: Entropy-based volatility prediction

Methodology

Data Collection: Trader actions (0=hold, 1=buy, 2=sell) Time Windows: Sequential trading periods Entropy Formula: H = -Σ(p_i * log2(p_i)) Testing Framework: Unit tests, robustness tests, market simulation, visual validation

Current Status

Research Phase: Theoretical framework validated with simulated data Next Steps: Testing on real market data from major exchanges Limitations: Has yet to be tested on real-time market data

Repository Structure

Shannon-Entropy/
├── data-collection.cpp          # Core entropy function
├── tests/                       # Comprehensive test suite
├── visualize_entropy.py        # Python visualization
├── requirements.txt            # Python dependencies
└── setup.sh                   # Environment setup

References

Shannon, C.E. (1948). “A Mathematical Theory of Communication”
Information Theory in Behavioral Finance
Market Microstructure and Entropy Analysis