In my experience from developing Backtest (https://backtest.curvo.eu), a backtesting tool for EU index investors, the hardest part has been finding good and reliable data that goes as far back in time as possible.
I only focus on index data so it's a bit easier but it's still hard. Lots of scraping, downloading and parsing CSVs, Excel sheets, some puppeteer magic...
Yeah, I'm trying to stay away from scrapping, parsing, and that whole mess. Right now I'm relying on Yahoo Finance undocumented API and possibly IEX Cloud in the future.
I only focus on index data so it's a bit easier but it's still hard. Lots of scraping, downloading and parsing CSVs, Excel sheets, some puppeteer magic...
Some sources that I use:
- MSCI data (https://www.msci.com/end-of-day-data-search)
- Financial Times website (e.g. https://markets.ft.com/data/indices/tearsheet/summary?s=SP5M...)
- Investing.com (e.g. https://www.investing.com/indices/us-spx-500)
- STOXX
- DAX
- FTSE for FTSE stock indexes (https://research.ftserussell.com/analytics/indexviewer/chart...)
- Yieldbook.com for FTSE bond indxes (https://www.yieldbook.com/m/home/index.shtml) - ICE website for ICE bond indexes
- ...
I'm constantly adding new indexes and sources (see https://backtest.curvo.eu/updates) but it's tough!
Some examples of what Backtest does:
- MSCI World: https://backtest.curvo.eu/portfolio/NoIgsgygwgkgBAdQPYCcA2AT...
- My portfolio: https://backtest.curvo.eu/portfolio/NoIgmg9gTghgdgcgM4AIAK0A...
If the author has any questions, just reach out to me! Happy to share my experiences.