I searched the net for script code that will download the iTunes Connect daily sales report but I came up empty. So I decided to write my own.
I've been interested in getting into Python programming and I thought what better way to learn Python programming than to write a script that will download the daily sales report from the iTunes Connect website. It turned out to be a fairly easy task to complete using Python. Guess it goes to show how powerful Python really is.
So here is my very first Python script. I'm sure there are things I could do better in the script, but hey, it's my first script. Enjoy.
UPDATE: The original code had a problem with retaining cookies set by the iTunes Connect website. The code below has been updated with the correct, working code.
UPDATE 2: The Python script below is now hosted over at Google. Just click here or visit http://code.google.com/p/appdailysales/ for the latest information and version.
posted by Kirby | 21-Oct-2008 12:50 PM | comments (10)
#!/usr/bin/python
#
# appdailysales.py
#
# iTune Connect Daily Sales Reports Downloader
# Copyright 2008 Kirby Turner
#
#
# This script will download yesterday's daily sales report from
# the iTunes Connect web site. The downloaded file is stored
# in the same directory containing the script file. Note: if
# the download file already exists then it will be overwritten.
#
# The iTunes Connect web site has dynamic urls and form field
# names. In other words, these values change from session to
# session. So to get to the download file we must navigate
# the site and webscrape the pages. Joy, joy.
#
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# -- Change the following to match your credentials --
appleId = 'Your Apple Id'
password = 'Your Password'
# ----------------------------------------------------
import urllib
import urllib2
import cookielib
import datetime
import re
urlWebsite = 'https://itunesconnect.apple.com/WebObjects/iTunesConnect.woa'
urlActionLogin = 'https://itunesconnect.apple.com%s'
urlActionSalesReport = 'https://itunesconnect.apple.com%s'
print '-- begin script --'
# There is an issue with Python 2.5 where it assumes the 'version'
# cookie value is always interger. However, itunesconnect.apple.com
# returns this value as a string, i.e., "1" instead of 1. Because
# of this we need a workaround that "fixes" the version field.
#
# More information at: http://bugs.python.org/issue3924
class MyCookieJar(cookielib.CookieJar):
def _cookie_from_cookie_tuple(self, tup, request):
name, value, standard, rest = tup
version = standard.get('version', None)
if version is not None:
version = version.replace('"', '')
standard["version"] = version
return cookielib.CookieJar._cookie_from_cookie_tuple(self, tup, request)
def showCookies(cj):
for index, cookie in enumerate(cj):
print index, ' : ', cookie
cj = MyCookieJar();
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
# Go to the iTunes Connect website and retrieve the
# form action for logging into the site.
urlHandle = opener.open(urlWebsite)
html = urlHandle.read()
match = re.search('action="(.*)"', html)
urlActionLogin = urlActionLogin % match.group(1)
# Login to iTunes Connect web site and retrieve the
# link to the sales report page.
webFormLoginData = urllib.urlencode({'theAccountName':appleId, 'theAccountPW':password})
urlHandle = opener.open(urlActionLogin, webFormLoginData)
html = urlHandle.read()
match = re.search('href="(/WebObjects/iTunesConnect.woa/wo/.*)"><img', html)
urlActionSalesReport = urlActionSalesReport % match.group(1)
# Go to the sales report page, get the form action url and
# form fields. Note the sales report page will actually
# load a blank page that redirects to the static URL. Best
# guess here is that the server is setting some session
# variables or something.
urlHandle = opener.open(urlActionSalesReport)
urlHandle = opener.open('https://itts.apple.com/cgi-bin/WebObjects/Piano.woa')
html = urlHandle.read()
match = re.findall('action="(.*)"', html)
urlDownload = "https://itts.apple.com%s" % match[1]
# Get the form field names needed to download the report.
match = re.findall('name="(.*?)"', html)
fieldNameReportType = match[3]
fieldNameReportPeriod = match[4]
fieldNameDayOrWeekSelection = match[6]
fieldNameSubmitTypeName = match[7]
# Ah...more fun. We need to post the page with the form
# fields collected so far. This will give us the remaining
# form fields needed to get the download file.
webFormSalesReportData = urllib.urlencode({fieldNameReportType:'Summary', fieldNameReportPeriod:'Daily', fieldNameDayOrWeekSelection:'Daily', fieldNameSubmitTypeName:'ShowDropDown'})
urlHandle = opener.open(urlDownload, webFormSalesReportData)
html = urlHandle.read()
match = re.findall('action="(.*)"', html)
urlDownload = "https://itts.apple.com%s" % match[1]
match = re.findall('name="(.*?)"', html)
fieldNameDayOrWeekDropdown = match[5]
# Set report date to yesterday's date. This will be the most
# recent daily report available. Another option would be to
# webscrape the dropdown list of available report dates and
# select the first item but setting the date to yesterday's
# date is easier.
today = datetime.date.today() - datetime.timedelta(1)
reportDate = '%i/%i/%i' % (today.month, today.day, today.year)
# And finally...we're ready to download yesterday's sales report.
webFormSalesReportData = urllib.urlencode({fieldNameReportType:'Summary', fieldNameReportPeriod:'Daily', fieldNameDayOrWeekDropdown:reportDate, fieldNameDayOrWeekSelection:'Daily', fieldNameSubmitTypeName:'Download'})
urlHandle = opener.open(urlDownload, webFormSalesReportData)
filename = urlHandle.info().getheader('content-disposition').split('=')[1]
filebuffer = urlHandle.read()
urlHandle.close()
print ' saving download file:', filename
downloadFile = open(filename, 'w')
downloadFile.write(filebuffer)
downloadFile.close()
print '-- end of script --'
Testing one, two, three. Just making sure comments are working. -KIRBY
posted by Kirby Turner | November 26 01:01 AMJust got my first app running on the App Store. So I will probably use the script too.
My reports are not available in the morning european time. so I need to figure out when the sales data is available.
I scheduled a cron job to download the daily sales report at 15:00 GMT -5 (3 PM Eastern). It's a little late in the day but I've never had a download problem even when the reports are delayed.
posted by Kirby Turner | November 28 08:27 AMVery useful thanks!
I couldn't get it to work in the terminal, won't execute without sudo and then nothing happens anyway... do I need to install Python or something? I have Leopard 10.5.5.
Right-click > build applet did the trick, but then I don't know if I can cron an app...?
If outputDirectory is '', it output to / instead of the current directory.
Auto-unzip fails if there is a space in outputDirectory. With '/Users/Ben/Desktop/iPhone/sales reports/', I got the following error:
/Users/Ben/Desktop/iPhone/sales.gz: No such file or directory
How about auto-naming the file with the date or something? Apple's name is a tad obscure ;)...
Cheers =)
Strange the script didn't run for you in the terminal. Have you tried using the -v verbose option? It will tell you what the script is doing as it runs. And be sure you are running the latest script available at:
http://code.google.com/p/appdailysales/
Not sure if you can cron the applet. I haven't tried that. I have a Ubuntu server that I use to run the script as a cron job.
I'll check out the unzip problem and I'll add an option to rename the file. Good suggestion.
Yep, using 1.5, verbose... do I have to explicitly call python? like 'python appdailysales.py'? I'm doing 'sudo ./appdailysales.py' right now... it asks for my password and then nothing.
Also, this morning it doesn't work anymore but maybe Apple's server is down or something?
I tried to copy-paste my console log but the comment is rejected :s...
Trying again... nope, no cigar. I'll just send the highlights:
report is not available. Please try again later.
execfile(os.path.join(os.path.split(__file__)[0], "appdailysales.py"))
line 222, in downloadFile
filename = urlHandle.info().getheader('content-disposition').split('=')[1]
AttributeError: 'NoneType' object has no attribute 'split'
@Benoit: Yes, depending on your setup you might need to use the command "python appdailysales.py -a appleid -p password" without the quotes. If you use sudo in front of the command that sudo will prompt you for your administrator password. sudo is probably not needed but of course I'm assuming your account has access to run the script. If it doesn't then yeah, sudo might be needed.
As for the "report is not available" message, did you include your apple id and password to the iTunes Connect web site on the command line call? If not then you will receive this message because without proper credentials there will be no report to down load.
I figured out that the error I got yesterday was because the report wasn't ready yet (they're usually ready by 10am but I guess it was late, maybe their system is lazy on mondays like everyone else).
"python bla.py" does the job... duh! (I baked the credentials into the file.)
No need for sudo either, I knew it was weird since the file is on my desktop so I obviously have access to it.
No HTML...gee whiz ;-) Nice to see you still have thecave. I worked with you at Post.net...
posted by SpeakerBoy | February 6 01:19 PMAdd Your Comment
