Accessing your Gmail inbox with Python

Written by Adrian Holovaty on June 18, 2004

Introducing gmail.py, a Python script that can log into a Gmail inbox and export messages as raw e-mail source, suitable for backup and import into other e-mail programs. It doubles as a Python interface to Gmail, so you can interact with your inbox from the Python command line (on a limited, read-only basis).

The product of a few nights' coding as a mental exercise, it's in its very early stages, but it seems to work for me. I haven't decided yet whether I want to take this on as a project or leave it as is. As it stands, you need to know Python to use it. Please let me know whether you're interested in seeing this go further.

I suspect this script will become obsolete as soon as Gmail starts offering export features of its own, but it's useful for backup purposes for now.

See also:

Comments

Posted by Michael D. Boyle on June 19, 2004, at 12:41 a.m.:

Sweet! Great work.

Now if there were only a way to import old mail into gmail I'd be golden.

Posted by James on June 19, 2004, at 1:19 a.m.:

Try Mark Lyon's Mbox & Maildir to Gmail Loader (GML).

Posted by pooya on June 19, 2004, at 6:43 a.m.:

Hi, What do you think about using gmail as a web hosting storage? I mean having all the big files in gmail's inbox and downloading them whenever needed and streaming it for the user.

is it possible?

Posted by Scott Johnson on June 19, 2004, at 8:38 p.m.:

It sure would be nice to see an official release of something like this from Google. PHP, Python, Java, and Perl bindings would be really nice. But I guess I can keep dreaming. I'm sure this is nowhere near their priority list.

Posted by Jorge Laranjo on June 19, 2004, at 9:52 p.m.:

Backup? You have 1GB of space...

Posted by Ian on June 21, 2004, at 8:51 p.m.:

I used your class to make a script that supposed to check gmail every 60 seconds to see how many new messages there are.

(there are tabs for the while statement, but theres apparently no fixed fonts allowed on these comments)

#!/usr/bin/python

import time

from gmail import GmailClient

c = GmailClient()

c.login('user','pass')

while True:

print len(c.get_inbox_conversations(is_unread=True))

time.sleep(60)

It always prints the same amount of unread messages, even if it changes after it starts. I don't really know python, so I imagine its some sort of gotcha with python (does the output of c.get_inbox_conversations(is_unread=True) get cached or something?). Any suggestions?

Posted by Adrian on June 21, 2004, at 10:45 p.m.:

Ian: It's always printing the same amount of unread messages for two reasons. First, the script doesn't mark messages as read when you retrieve them, mostly because this is intended to be a backup script rather than a full, read-write programming interface. Second, yes, it caches that output anyway.

Have fun learning Python. You'll never go back. :-)

Posted by Ian on June 22, 2004, at 5:43 p.m.:

Thanks, I'll have go ahead and try breaking the cacheing feature. Basically I want to not have to login for every inbox check. I think I might end up with some trouble when I login using a browser when the script is on. But I'll cross the bridge when I get there.

Posted by Ian on June 22, 2004, at 6:12 p.m.:

5 line change: http://www2.truman.edu/~iam504/gmail_cacheoption.py

Though now that I think about it, the 'correct' way would be an attribute of the class. But this is good enough for what I'm doing with it.

Posted by Ian Monroe on June 23, 2004, at 4:44 a.m.:

Made a PyKDE program that shows the number of new messages in the inbox in the KDE system tray. I don't really know python or KDE, so its more hassle then its worth to fix up the 'ghost window' that goes along with it.

http://www2.truman.edu/~iam504/KNewGmail.py

Posted by Gustavo Sverzut Barbieri on June 24, 2004, at 1:06 a.m.:

Small utility to migrate yahoo contacts to gmail.

Uses Yahoo! CSV (comma separated).

Notes:

- I ripped a small subset of GmailClient(), just the necessary to add contacts

- As I have a yahoo.com.br account, I get the CSV in brazilian portuguese. I added the possibility to map values in CSV to class YahooContact attributes, so if someone have an account in other country, please mail me the CSV header.

Get it from: http://gsbarbieri.sytes.net/export_contacts-yahoo_to_gmail.py

Posted by anonymous on June 27, 2004, at 2:02 a.m.:

Does this script work anymore ? It seems like I always get the error "Gmail might have redesigned" on login. Can anyone confirm if they can still login, please ?

Posted by Adrian on June 27, 2004, at 2:53 a.m.:

Anonymous: I just tried it, and it worked for me.

Posted by anonymous on June 27, 2004, at 3:32 a.m.:

Same anonymous here:

I've stepped through with a debugger and it appears that the following is happening:

within the login method, when there is a re.search for a "var js_version\s*=\s*'([^']+)", I get the contents of c as being

"<script>top.location="http://gmail.google.com/?dest=http%3A%2F%2Fgmail.google.com%2Fgmail";</script>" instead (so it obviously fails the match) and throws LoginFailure.

Can anyone tell if what's going on ? I suspect that my machine is forcing Google to take a HTTPS route ? or is it something different ? I downloaded the script 5 minutes ago and tried again.. but it's not working either.

Posted by anonymous on June 27, 2004, at 3:37 a.m.:

Not that anything vital depends on me getting this to work *grin* or anything like that. I'm just curious to see it in action and maybe (just maybe) I can extend it.

Certainly, I'd like to put a wxPython wrapper around it if I can

Posted by Adrian on June 27, 2004, at 3:40 a.m.:

Anonymous: Which OS are you on, and which version of Python are you running? Perhaps your Python installation has a slightly different version of the urllib or urllib2 modules. (I've heard, for instance, that Mac OSX has differences.)

Posted by anonymous on June 27, 2004, at 3:45 a.m.:

Win32, Python 2.3.2

Hmm, I apologize for adding noise to your comments. This is probably a machine specific cookie or setting causing the problem.

Posted by anonymous on June 27, 2004, at 4:17 a.m.:

Confirmed, it is a problem with my machine/Python installation.

cygwin, Python 2.3.3 works fine. I just tested it out. Leaving this message in case anyone else has the same problem.

Posted by battez on June 28, 2004, at 1:31 p.m.:

great tool.

Posted by Brian Swallow on June 30, 2004, at 1:58 p.m.:

What about SENDING mail via gmail and the class? I would find it useful.

Posted by Hendrik Mans on June 30, 2004, at 5:11 p.m.:

Proxy support?

Posted by Hendrik Mans on June 30, 2004, at 5:42 p.m.:

Okay, figuring out proxy support in urllib2 wasn't so hard, but I still can't actually access the gmail.com HTTPS URLs through our proxy here (using urllib2):

socket.sslerror: (1, 'error:140770FC:SSL routines:SSL23_GET_SERVER_HELLO:unknown protocol')

Am I missing something obvious?

Posted by justin on July 1, 2004, at 11:14 a.m.:

somebody, somewhere will integrate this script into SuperKaramba.

Posted by anonymous on July 1, 2004, at 10 p.m.:

so how do I hack this to delete all my messages in GMail? I see a "delete contact" option, but not a "delete message" or "delete all messages" option...

And no, I don't want to use the "nuke" option in Gmail that kills everything. I want to keep the account, just clean house on the several thousand msgs I have there.

Posted by Andy Triboletti on July 2, 2004, at 12:14 a.m.:

This isn't working in OS X with Python 2.3.3, looks like an issue with urllib2.

I'm trying to figure it out, but haven't been successful yet.

>>> from gmail import GmailClient

>>> c = GmailClient()

>>> c.login('triboletti', '<censored>')

Traceback (most recent call last):

File "<stdin>", line 1, in ?

File "/Users/andy/python/gmail.py", line 95, in login

post_data="continue=http://gmail.google.com/gmail&service=mail&Email=%s&Passwd=%s&submit=null" % (username, password))

File "/Users/andy/python/gmail.py", line 192, in _get_page

f = urllib2.urlopen(req)

File "/sw/lib/python2.3/urllib2.py", line 129, in urlopen

return _opener.open(url, data)

File "/sw/lib/python2.3/urllib2.py", line 326, in open

'_open', req)

File "/sw/lib/python2.3/urllib2.py", line 306, in _call_chain

result = func(*args)

File "/sw/lib/python2.3/urllib2.py", line 901, in http_open

return self.do_open(httplib.HTTP, req)

File "/sw/lib/python2.3/urllib2.py", line 895, in do_open

return self.parent.error('http', req, fp, code, msg, hdrs)

File "/sw/lib/python2.3/urllib2.py", line 346, in error

result = self._call_chain(*args)

File "/sw/lib/python2.3/urllib2.py", line 306, in _call_chain

result = func(*args)

File "/sw/lib/python2.3/urllib2.py", line 472, in http_error_302

return self.parent.open(new)

File "/sw/lib/python2.3/urllib2.py", line 331, in open

'unknown_open', req)

File "/sw/lib/python2.3/urllib2.py", line 306, in _call_chain

result = func(*args)

File "/sw/lib/python2.3/urllib2.py", line 914, in unknown_open

raise URLError('unknown url type: %s' % type)

urllib2.URLError: <urlopen error unknown url type: https>

Posted by DyliLama on July 14, 2004, at 6:28 p.m.:

hi,

love the script. One question....I have 2 gmail accounts. I just opened the 2nd and want to transfer every email from my first (as in all archived emails) to the 2nd. Any idea how to go about doing that?

thanks.

Posted by anonymous on July 14, 2004, at 9:27 p.m.:

i have the same problem as the person above... 2 gmail accounts, want to transfer all mails to the other. any ideas? thanks

Posted by anonymous on July 14, 2004, at 9:27 p.m.:

i have the same problem as the person above... 2 gmail accounts, want to transfer all mails to the other. any ideas? thanks

Posted by Preston on July 30, 2004, at 7:22 p.m.:

Please forgive me if I'm being dense, but how do I backup my ENTIRE gmail account using this python script? I looked at the API and it looks like it only gives access to the converations in the "Inbox".

What I'm interested in doing is setting up a weekly cron job to backup the whole enchilada...

Thanks!

Posted by Adrian on August 1, 2004, at 6:30 a.m.:

Preston: If you're familiar with Python, you should be able to alter the script to retrieve non-inbox folders. Take a look at the _get_message_stubs() method, which takes the folder name as an argument.

I'll add the functionality myself when I get a chance.

Posted by Grangin on August 7, 2004, at 5:16 a.m.:

Hey I've been using this python script to monitor gmail accounts from gkrellm (tut for doing that on that site there, which I don't really own), and I'm currently working on patching mailwatch to allow for monitoring of multiple gmail accounts. However I just tried to use the python script to logon (was working up until recently) and as of 08/05/04, the login doesn't work anymore. It looks like google changed the procedure for logging in on that AuthBox page that was used in the python script. It borks right after the part with the post data etc.

I suppose I can try and patch it but I have no idea where to begin. But that's part of the joys of programming isn't it!

This python script rules btw, gmail wouldn't be what it is without the ability to monitor in gkrellm.

Here's the error output, though I imagine it happens to everyone:

Traceback (most recent call last):

File "/home/tristan/.gkrellm2/fetchgmail/fetchgmail.py", line 41, in ?

gmailchk.get_mail_thread()

File "/home/tristan/.gkrellm2/fetchgmail/fetchgmail.py", line 23, in get_mail_thread

mailclient.login(c_user, c_pass)

File "/home/tristan/.gkrellm2/fetchgmail/gmail.py", line 110, in login

raise LoginFailure, "Wrong username or password."

gmail.LoginFailure: Wrong username or password.

Posted by Grangin on August 7, 2004, at 5:20 a.m.:

Ooops, actually that's 08/06/04, today, sorry, my clock in xfce4 is dumb and doesn't update the date in the tooltip (cvs).

Posted by Grangin on August 7, 2004, at 6:02 a.m.:

More info after digging.... Looks like the page has completely changed. Whereas the old Auth page would store the cookie information in the actual source, it looks like it is now all handled behind the scenes. Looks like gmail is trying to keep us out (I wonder if they're specifically targetting this type of thing ??? Why... ;_;)

From the info on the pages now, it looks like it's impossible to generate the cookie with python. Looks like an entirely new method is going to have to be devised... if that is even possible.

This is sad though. I really hope you geniuses out there can figure something out! (or maybe our combined brain powers will be enough =)

Posted by anonymous on August 13, 2004, at 4:59 p.m.:

If you want an alternative Python interface that works with the login changes (in CVS currently) and offers proxies for SMTP, FTP and POP3 (in CVS currently) and archiving ability you could look at libgmail:

http://libgmail.sf.net/

--Phil.

Posted by Grangin on September 7, 2004, at 4:12 a.m.:

Wow thanks for the info, I'll look into it.

Posted by Paul on September 26, 2004, at 2:20 p.m.:

Is there any way to get the sender of an email with gmail.py?

Posted by anonymous on October 1, 2004, at 10:08 a.m.:

urllib2.URLError: <urlopen error unknown url type: https> - solution

if you encounter this, you're probably missing ssl.

I found this library for windows at: http://www.junkheap.net/projects/python-windows-ssl/python-windows-ssl.html

http://www.junkheap.net/projects/python-windows-ssl/python-windows-ssl.html

Posted by Andy Triboletti on October 4, 2004, at 6:57 p.m.:

anonymous:

yeah, I found that too- but I'm looking for something for Mac OS X. I know I can replace Apple's Python with a custom one, but I was looking to package this up as an application..

Posted by Manish Jain on October 8, 2004, at 1:14 p.m.:

How will it work ????

I was unable to use it in SuSE 9.1

Contact me at manish.jain.85@gmail.com

the errors received are :

Please wait, logging in...

Traceback (most recent call last):

File "libgmail.py", line 805, in ?

ga.login()

File "libgmail.py", line 255, in login

pageData = self._retrievePage(req)

File "libgmail.py", line 280, in _retrievePage

resp = urllib2.urlopen(req)

File "/usr/lib/python2.3/urllib2.py", line 129, in urlopen

return _opener.open(url, data)

File "/usr/lib/python2.3/urllib2.py", line 326, in open

'_open', req)

File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain

result = func(*args)

File "/usr/lib/python2.3/urllib2.py", line 908, in https_open

return self.do_open(httplib.HTTPS, req)

File "/usr/lib/python2.3/urllib2.py", line 886, in do_open

raise URLError(err)

urllib2.URLError: <urlopen error (-3, 'Temporary failure in name resolution')>

Posted by d4rkstorm / hellrazor on January 21, 2005, at 1:47 a.m.:

Thought i would add an Update to the forum , I would strongle suggest aving a quick search for "GmailFSv1 or v05" , it allows complete mounting of the 1,2,3fif (however many you run it on) , makes them local-Hardrivess..so for each email acct on gmail, you will have an extra (although not sure how many can run at once) , successfully working now , i have link here but just google it... is quite good.

As for some question i saw about making all mail goto one account..

In each email , there will be a setting to "send any incoming messages to <blah@mail.com> ? IF yes, please enter alternate email" , set that with the 2-3 accts (ie- i have 3 Gmail accounts, i make all 3 send to ONE,wich i use as my main login ,i no need for s script etc,just use Gmails settings and presto!

Any extra Info/questions..drop a line .. nonet.team@gmail.com

Posted by anonymous on March 29, 2005, at 7:51 p.m.:

Hello, I'm having the following problem

urllib2.URLError <urlopen error (2, 'the operation did not complete (read)')>

(Using windows xp, Python24)

..Note that support for ssl is included in python 2.4

Anyone have any suggestions?

Posted by Hasan Diwan on April 10, 2005, at 11:49 p.m.:

Hasans-iBook: 2:44pm %python2.4 [~/Developer/GMailRSS]

Python 2.4 (#1, Feb 19 2005, 13:02:16)

[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin

Type "help", "copyright", "credits" or "license" for more information.

>>> import gmail

c=gmail.>>> GmailClient()

>>> c.login('hasan.diwan','password')

Traceback (most recent call last):

File "<stdin>", line 1, in ?

File "gmail.py", line 110, in login

raise LoginFailure, "Wrong username or password."

gmail.LoginFailure: Wrong username or password.

>>> c.login(r'hasan.diwan%44gmail.com',r'password')

Traceback (most recent call last):

File "<stdin>", line 1, in ?

File "gmail.py", line 110, in login

raise LoginFailure, "Wrong username or password."

gmail.LoginFailure: Wrong username or password.

>>> c.login(u'hasan.diwan@gmail.com', u'password')

Traceback (most recent call last):

File "<stdin>", line 1, in ?

File "gmail.py", line 110, in login

raise LoginFailure, "Wrong username or password."

gmail.LoginFailure: Wrong username or password.

This is with python 2.4 and python 2.3. Please advise.

Posted by NNnaji on August 15, 2005, at 1:52 p.m.:

Having following difficulty

urllib2.URLError <urlopen error (2, 'the operation did not complete (read)')>

(Using windows xp, Python24)

..Note that support for ssl is included in python 2.4

Posted by hammad on October 7, 2005, at 10:12 p.m.:

i dont know pyton enougf

can any body tell me how i can make this script that it checks inbox after every 1 hour , retrive new , save them in mysql database , and after saving delete them from inbox

plz any body help me i really need this type of script

also i have and intrested idea about it , intrested coder can mail me at

hammadfahim@gmail.com

Posted by Sanjay Arora on February 3, 2006, at 5:54 p.m.:

Would it be possible to programmatically delete all mails in trash or to delete mails in spam? Is this possible with libgmail or any existing api's usable on linux...would prefer python! Any pointers to existing code? Am a python newbie....and use getmail for getting pop3 mail from gmail.

Problem is that have subscribed to some groups with heavy mail and need to delete trash because gmail seems to count trash mail in the 2 GB disk quota.

Hope someone can help.

With best regards.

Sanjay.

Posted by Remco H. on February 14, 2006, at 8:06 p.m.:

How do you use the script?I typed "python /home/remco/gmail.py (this is the file where i put the script in..) but no output...nothing happens...

I have Python 2.4.1 on Suse Linux 10 but i don't understand how python works...

Can anyone please explain what i'm doing wrong?

Posted by orkan on March 20, 2006, at 4:19 a.m.:

hi, I am also getting the wrong username and password error. (in windows XP and python2.3)

any help?

Traceback (most recent call last):

File "<stdin>", line 1, in ?

File "gmail.py", line 110, in login

raise LoginFailure, "Wrong username or password."

gmail.LoginFailure: Wrong username or password.

Posted by bajzel on May 6, 2006, at 3:08 p.m.:

Hi.

I'm trying to send html based message using libgmail and I'm a little confused.

I can send text-based messages and this works fine but with html-based it's not so simple I suppose, because there must be set up some additional things... I've tried to look into libgmail.py but without result.

All I need is simple example which sends html-based message using libgmail.

Thanka in advance,

bajzel

Comments have been turned off for this page.