A Quick Hack to Get Historical Data from Yahoo!Finance

If you need financial data for your softwares, here is a quick hack in Bash (runs on my mac and I suppose on linux too) to get quotes from yahooFinance.

This script download a .csv file containing the historical data from today to the day of your choice for each stock listed on the S&P 1000.

First, you need to download the .csv file containing the symbol of every stock listed on the S&P 1000. This file can be found here. In my case, the file is named 17-FEB-1009_1000.csv.

Then, copy the following to a file called yahooDown.sh :

#!/bin/bash
# prices from S&P500 can be downloaded from
#http://www2.standardandpoors.com/portal/site/sp/en/us/page.topic/indices_500/2,3,2,2,0,0,0,0,0,2,3,0,0,0,0,0.html

# We download from today to the date in the command line

#./yahooDown.sh standardPoorList.csv DestDir 1-1-1995
# e.g. ./yahooDown.sh 17-FEB-2009_1000.csv SP1000 1-1-1995

todayDate=$(date "+%d %m %Y")

# Start date
sDay=$(echo $todayDate | awk '{print $1}')
sMonth=$(echo $todayDate | awk '{print $2}')
sYear=$(echo $todayDate | awk '{print $3}') 

# end date

eDay=$(echo $3 | sed "s:-: :g" | awk '{print $1}')
eMonth=$(echo $3 |sed "s:-: :g" | awk '{print $2}')
eYear=$(echo $3 |sed "s:-: :g" | awk '{print $3}') 

# command to fetch data the CSV files from yahoo

DESTDIR=$2
WGET="curl"
OPT1="-o"

COM1="http://ichart.finance.yahoo.com/table.csv?s="
COMd="&d="$(expr $sMonth - 1)
COMe="&e="
COMf="&f="
COMg="&g=d"
COMa="&a="$(expr $eMonth - 1)
COMb="&b="
COMc="&c="
COMlast="&ignore=.csv"

# the command we pass to wget
for i in $(sed -e "s:,: :" -e "/Symbol*/ d" -e "/^$/ d" $1 | awk '{print $1}'); do

echo "Downloading the data for symbol " $i
WEBURL="$COM1$i$COMa$COMb$eMonth$COMc$eYear$COMd$COMe$sMonth$COMf$sYear$COMg$COMlast"
COMMAND="$WGET $OPT1 "$DESTDIR/$i.csv" "$WEBURL""
$COMMAND

done


Now you need to turn it into an executable file. Open a terminal in the directory where the file yahooDown.sh is located and type:
$ chmod +x yahooDown.sh
Now, create a directory where you want the .csv files of each stock to be placed:
$ mkdir SP1000
Now, just type:
$ ./yahooDown.sh 17-FEB-2009_1000.csv SP1000 1-1-1998
Just wait a bit… You should see something like this:
$ ./yahooDown.sh 17-FEB-2009_1000.csv SP1000 1-1-1990
Downloading the data for symbol  COMS
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  217k    0  217k    0     0  60340      0 --:--:--  0:00:03 --:--:-- 65004
Downloading the data for symbol  NDN
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  147k    0  147k    0     0  26304      0 --:--:--  0:00:05 --:--:-- 35161
Downloading the data for symbol  AHC
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10817    0 10817    0     0  24368      0 --:--:-- --:--:-- --:--:-- 43963
and voilà! The historical data of every stock listed on the S&P1000 is now ready to be used in the directory SP1000. Here is a (partial) listing of the same directory on my machine:
$ ls
AAI.csv		CCMP.csv	FIC.csv		KND.csv		ORLY.csv
AAP.csv		CCRN.csv	FIF.csv		KNDL.csv	OSG.csv
ABAX.csv	CDI.csv		FINL.csv	KNOT.csv	OSK.csv
ABCW.csv	CDNS.csv	FL.csv		KNSY.csv	OSTE.csv
ABFS.csv	CDR.csv		FLO.csv		KNX.csv		OXM.csv
ABM.csv		CEC.csv		FMBI.csv	KOPN.csv	OXPS.csv
ACAP.csv	CECO.csv	FMC.csv		KRC.csv		PACW.csv
ACAT.csv	CELL.csv	FMER.csv	KRG.csv		PALM.csv
ACF.csv		CEM.csv		FNF.csv		KSU.csv		PAS.csv
ACI.csv		CENTA.csv	FNFG.csv	KSWS.csv	PBKS.csv
ACIW.csv	CENX.csv	FOE.csv		KWK.csv		PBY.csv
Okay, that’s all folks.