Saturday, February 6, 2010

Replacing Microsoft Money

I recently put together a discussion of some work I became interested in while investigating options for either keeping or Replacing Microsoft Money . Since I don't yet know how to create a useful comments / suggestions box on Google Sites (or if it's even possible), I decided to put an entry here to accept comments and to discuss.  The scripts themselves are located on a Google Sites entry.

I think it took me longer to write about the changes and figure out how to put it on a web site than I spent actually writing the code!

NOTE:  It is preferred that you comment using a Google account, since you can then delete a post.  Posts cannot be edited, but Google users can delete and repost (I think?).


  1. The first comment I've received regarded the copying of site information from a prior version of the OFX Python scripts. While modifying, I had added a field named "minInterval" that is intended for sites where you must use a download interval of some minimum length.

    I thought about this some more and decided that the field should be optional. I modified the code so that you can now define a site *without* a minInterval field and the code doesn't care.

    Now it just uses the defaultInterval value for those accounts (as it should), so Site definitions from other versions should work without modification.

  2. Great job Bob!! I was going to rework the code to make minInterval optional, as in most cases we'll probably use 1. Very small typo in line 7: doit = raw_input("Download transactions? (Y/N) ")

    I'm assuming that when we want to update your scripts, we extract all but ~ Do we need to delete the compiled python files (pyc)? or do they recompile themselves?

  3. Thanks for the typo catch! You are correct that you will want to keep your existing file across updates. I thought about that, and may even rename to something else, with some instructions to rename it for the first install. For now, extract everything *except* when updating to a newer version.

    The compiled Python files (.pyc) will automatically recompile when new versions are introduced.

  4. If I might make a suggestion, just have your setup program make a copy of so that the user can paste their info back in if the file is overwritten.

    I found out the hard way that you can't change the site name from small characters to CAPs after you've run setup either.

  5. You're thinking like me! The caps issue is the same as before, since the original scripts use case sensitive Site structures. I didn't (and don't) care for that, but haven't addressed it either. I'm thinking that all Site entries should be forced to upper case, as I don't see any reason not to. I should probably look at it.

    The real culprit is user-editing of the control info in it's current form. It uses a Python data structure that has little forgiveness for caps, commas, quotes, etc. I almost replaced it with a loose text structure with a "parser", but that takes more time and I didn't know if many other folks would use it.

  6. Re: a backup control file, I was leaning towards using a template file in the distribution. A setup::backup would work, but I could see someone downloading a new version, running setup, and overwriting the backup with a new version accidentally. Maybe a .\template\ file that never installs to root, but is only used during initial setup? Something to consider...

  7. You've done an excellent job Bob; my comment about the CAPs wasn't a complaint - just an observation. I started adding .upper() to the sites callouts in the scripts, but changed my mind :)

    In case you aren't aware, I'm promoting your Blog in the new blog I started a few days before yours. Yours is so well done, I don't really need to continue mine ;) (You might be interested in the currency exchange rates and market indexes that can be added in the

    -Dan (c230, ameridan)

  8. Thanks much Dan. Wow, I had no idea. If I had known you were doing all of that we could have tag-teamed! I'll look at the exchange rate/index info sometime soon. FWIW, I think I could have written an entire Java GUI version of the code in less time than trying to *describe* the work :)

    BTW, I added a few lines to create up to 3 backup copies of when Setup is run.

  9. One other thought I had for

    Although it probably doesn't matter now, I've been using (Quicken 2009):

    config["appid"] = "QWIN"
    config["appver"] = "1800"

    instead of (Money Plus):
    config["appid"] = "Money"
    config["appver"] = "1700"

    because it might raise a flag at the banks once Money servers go offline as to how Money is requesting the OFX download.

  10. Good idea. I had previously used QWIN, but had changed to "Money" while troubleshooting Discover. I'll put it back to QWIN just in case.

  11. I edited the routine again... since I hadn't implemented the backup very well. The new distribution that I've just uploaded does a couple of things:

    1. is no longer included. Instead, there's a file named control.template. Setup will create a new file from the template *if* doesn't already exist.

    2. A backup copy of the is created by setup, but *only* if the file has been altered since the last backup was created.

  12. There was a glitch in the file I downloaded last night. Distribution updated. I highly recommend that anyone using the code download the latest version.

  13. Very nice strategy! and with backwards compatabilty too. I am enjoying learning from a pro. Your site format works much better than my Blog format too, but I thought this OFX script strategy was going to evolve slowly - not have one guy come in and clean the whole thing up!! :)

  14. I decided that I *really* don't care for defining the Site and Fund structures as Python code. Doing so is rife with issues, including:

    1. It's code, so it gets compiled. Anything a user does that's even slightly off will cause the whole thing to crash.
    2. It's sensitive to everything that Python is, including type case, indentations, etc. (tabs are not allowed, for example).

    The fix is to separate all user entries into a non-Python text file, and then parse the entries into their correct Python representation. I have 98% completed this task, but I'm not sure whether to provide backwards compatibility w/ the existing method. Doing so makes things a bit messy to me. Comments?

  15. I noticed that yesterday with TFB's script comments. I think it was Scott that had one extra comma causing the script to error out. I would agree that it is pretty sensitive right now. The scrubber didn't work for me right away, because my site url came from a different source that had a / at the end.

    If you can see how many have downloaded your zip file, you may determine that not too many have done so yet. Anyways, feel free to proceed - I'd be willing to redo my setup file. :)

  16. I extracted the Money icon out of the EXE if that helps you in your next iteration (available on my site). Regarding the Java GUI you mentioned, I felt the one on OFXHome (OfxGet) is too restrictive with pull downs in that the website admin has to do all of the work; the user is not able to try sites that aren't already defined. For your sake, I hope you aren't going to that extreme.

  17. Regarding the scrubber issue, I just changed the trigger line for Discover to the following:

    if 'DISCOVERCARD' in siteURL.upper():

    Would that have worked for your case?

    I don't intend to implement a Java version of this anytime soon. Actually, getting the user input into a text file (read-in by a Python parser) will probably be as far as I go. If it "works" and isn't too difficult to implement, even a non-programmer should find it useful.

  18. I should also mention that I used this an excuse to learn Python :) Python is powerful, but it's not a very good way (IMHO) to implement a user-friendly interface, which is why I don't see taking it too much further in this form.

  19. Yes, "if 'DISCOVERCARD' in siteURL.upper()"
    would have worked for slight variations. Personally, I think your package is great already, so your last bit of tweaking will just polish it off.

  20. I take it you're going the user will define SITE in, rather than reading it into setup from modified Saving site data into ofx_config.cfg and instead? Wouldn't need to even have the template then / or bother with backup copy.

    When testing SITE without account # parameter, account # data is provided - it would be kinda neat if you could feed the array with that (especially if there are multiple accounts) for setting up the individual accounts.

  21. There will be a sites.dat file, which is just a text file that the user edits. Site, fund and stock info will be defined in a loose XML structure. The parser will ignore tabs, spaces, commas, comments, etc. (things the Python compiler does not ignore) and force the formatting to something that is internally consistent. There is "case sensitivity" to site field formatting, since the data for a site is often case sensitive (e.g., the URL has to match case for a lot of servers), so matching with the OFX site database will be the user's responsibility... no way around it.

    If you don't mind, describe the last comment a little differently. I'm not sure I understood correctly. Thank you!

  22. I just noticed something in the original OFX script. It passes the FIORG field as the BROKERID for type=INVSTMT requests, yet actual BROKERID fields are defined differently for some investment institutions. It doesn't seem to matter for Vanguard, as it's working with the BrokerID=fiorg, but now I wonder if the Site data should explicitly include a BrokerID field when available?

  23. Never mind on my last paragraph - that was more of a wishlist item. It probably doesn't fit your scheme anyways.

    Here is what I had in mind though: User clicks on setup and finds SITE data. Then enters userID and password. Then runs Test. The DOS screen would list the account numbers to make the next setup step easy (copy and paste) **OR** if you are writing all of this data into an array in ofx_config.cfg - feed the account #s in as well.

    For a credit card account it would save typing in the 16 digits. In my case at Schwab where I have 6 different accounts, it would make setup of the individual accounts easier too.

    Now the user can test each site/account entry individually, as you recommend they do currently.

    ??Clear as mud now?? :)

    Like I said, it's just a wishlist item, but please don't feel compelled to implement. Your current scheme is just fine.

  24. From what I've read, it's better to provide all of the data, and if not needed it is just ignored. The same applies to bankID, fiorg, fid, etc.

    The last post I did is the process Money uses, but please don't implement if it doesn't fit your plans.

  25. I had assumed that sites would ignore extra fields, but I learned differently. When testing the new parser, the Vanguard connections were failing with a simple "Error 400" message. I had the parser assigning a value to every known field for each site (even if empty), and had added a blank FID placeholder to the Vanguard entry (not sure why, but I did). That was the culprit. If Vanguard sees a null FID field, it rejects the request without fanfare. I don't know what would happen if it had a fid value, since Vanguard doesn't use the field. I'm assuming it would do the same.

    I'm making the assumption now that only those fields defined by a site should be presented to their server, so no more NULL field entries for me. I'll probably add some logic to the parsing sequence to skip null entries, in case others do what I did.

  26. Interesting...

    The settings that work for me at Schwab:
    “caps”: [ "SIGNON", "INVSTMT" ],
    “fiorg”: “ISC”,
    “fid”: “5104″,
    “url”: ““,

    The GNUCash settings:
    FID: 5104
    ORG: ISC
    Broker ID:

    and the OFXHome settings:
    Broker ID: SCHWAB.COM

    Quite a differnce; I'll have to experiment to see what other combinations work.

  27. Dan,

    If you have a chance, could you beta test the latest version? Download @

    Place into a new directory, run setup, and then edit the sites.dat file for one of your institutions and see how it goes.

    Also, you can backup your file (for good feeling) and place the new code into the same directory as your current setup. It will use the existing file if it exists *rather* than the sites.dat file (i.e., it defaults to the "old" way), so the file has to disappear before the sites.dat parser kicks in. So... it should be "backwards" compatible... although I don't think many are using it at this point.

  28. Robert,

    I am happy to report that your package works exactly as described. Now that I've cleaned up my Site data, I'll use that rather than my old

    The data that ended up working for Schwab:

    SiteName: SCHWAB
    AcctType: INVSTMT
    fiorg : ISC
    url :

    which doesn't look much like OFX Home info, but that has nothing to do with your great scripts. Their log that shows those settings working is deceiving though. And does Broker ID ever get used?

    Anyways Beta > Release :)

  29. Robert,
    Thought I would point out that xxx has written a Java app last month that you might want to look over. See his comment on my Blog.

  30. Thanks Dan! I've polished off the new version, and will post it as soon as I have time to edit the install instructions. Tied up with real work right now...

  31. Robert,

    See if your Beta generation or work today. I get this:

    Traceback (most recent call last):
    File "", line 33, in
    File "C:\Documents and Settings\...\", line 205, in getOFX
    dtstart = time.strftime("%Y%m%d",time.localtime(time.time()-interval*86400))

    TypeError: unsupported operand type(s) for -: 'float' and 'str'

  32. Deja vu from yesterday. I recall that error while debugging yesterday, so I think I fixed it.

    Try the latest snapshot and let me know:

  33. Back in biz!

    Didn't work until I renamed my to but I like my new dat file better anyways. I was just keeping it around, but the old and new shouldn't be mixed. No one else will even have and I reintroduced into the folder after the fact.

  34. Robert,

    Bug? was always live in that if you want to add another bank site, would see it. If I add another bank site in sites.dat now, setup does not see it when I go to add account.

  35. Never mind; I forgot

  36. Wish I could edit/delete comments...

    I had forgot the /site after the data

  37. Thanks Dan! The old should live happily with the new system, so I consider that a small bug that I'll look at before posting the new version live. Yup... site info must fall within the (site.../site) block, since that's the only thing flagging the entries as sites.

    If you use a Google account you can delete comments. I *thought* that using any of the supported user groups (wordpress, etc.) could delete comments too...? Still learning how this site works, but not spending much time trying. I'm going to check to see if there's a setting somewhere.

  38. Found the bug... but it wasn't so small, and only mattered for the old file. I fixed it anyways, since it offered a good learning experience. Feel free to grab the latest snapshot at the same link above, since it also fixes a few other potential "gotchas".

  39. Robert,

    Let me bounce one other idea I have off of you:

    (Kind of a Debug value for quotes seperate from the transactions debug)

    If a user would like the stock quotes to be closing bell prices and debug is off and he runs the scripts in the middle of the day, those quote values are it for day (unless they are manually erased prior to running the scripts again).

    Do you think it would be too confusing to ask the user after "Download transactions? (Y/N)"
    - something to the effect "Do you plan on updating quotes again later today? (Y/N)"?

    If the user answers Yes, he would see the CMD screen of values and even generate the OFX file, but the quotes wouldn't be sent to Money; just the transactions / statements.

  40. The more I think about it - scratch that last idea. If I'm not mistaken, the quotes downloaded from your brokerage accounts probably work the same way (1st in for the day are final) and if that's the case, then if you really would like closing bell prices for your stocks and funds, then you probably need to run the scripts in the evening.


  41. Instructions updated and latest snapshot uploaded.

  42. Not sure what the other "gotchas" were, but all seems to be good now with the residing in the folder. My site names were lower case in and I like how you CAP them in setup to add accounts.

  43. One last update... I had updated the revision # to be a string, and messed it up. Just now replaced the file...

  44. I was experimenting and replying to the package just prior to your production release, but it looks like you just tweaked subsequent to my download this morning. Great job on the instructions too - I think you've got everything covered. :)

  45. The other "gotchas" were mostly minor, but one related to some possible CaSe issues. I had not consistently enforced case throughout, but it should be now. It was fine for the site.dat method and completely new installs, but not for an existing ofx_config.cfg. I think it's consistently enforce now throughout.

    A note: The SiteName field *value* is forced to uppercase, but all other field values are mixed case, since some fields *require* it (such as some URLs). Field *names* are forced to uppercase regardless, but not the values.

  46. "The SiteName field *value* is forced to uppercase"

    This is great - that is what messed me up originally as my old has SiteNames in lower case.

    One note to readers regarding Discover downloads - although the transactions will no longer replicate using this script method (thanks to the great scrubber routine), if you run these scripts and do online updates as well (this isn't normal, but some of us want to gain confidence in the Python script method prior to halting the online updates), the transactions won't be matched together as the transaction IDs still must differ slightly between the two methods. Simply choose the "Change" option to manually match the duplicates together and all is well.

  47. Robert,

    One recommendation in your Get Scripts page...

    Instead of saying: If you want to download quotes for stocks and mutual funds to Money that you do have in any other accounts, then you will want to create a special "dummy" account to receive the quote statement.

    I'd reword it to something like: If you want to download quotes for stocks and mutual funds to Money that are not in your brokerage / bank portfolio(s) amd therefore are not being updated either online or via OFX, then you will want to create a special "dummy" account to receive the quote statement.

  48. Good comment Dan. I edited the instructions a bit more, since I was a bit terse the first time around. Maybe it reads better now?

  49. Yes. Just helps to avoid confusion for future users :)


    I added this to my (along with the menu entry: print "8. Test Stock/Fund Quotes" )

    elif menu_option == 8:
    #test stock/fund quotes

    It makes the setup program a nice way to run all available specific scripts now.

  50. In trying to keep the menu "short", I included that feature when a user selects "Enable Stock/Fund Quotes". So, disabling and then re-enabling asks "Do you want to test...". I almost put a "utilities" sub-menu for additional options like this, plus a few more, but refrained.

    BTW, when you run, you are asked whether to continue "y/n". If you enter "r", the script will run the same as a Yes answer, but it will prompt to continue at the end (in case you want to review the screen contents before the command window closes).

  51. I had tried "D" based on the comment in the script; didn't see the "R" further down. Thanks.

    Not being familiar with Java, let me ask a dumb question... "The only catch is that it needs to be distributed as open-source code, not compiled. Afterall, who is going to entrust their account information to a black box from someone they don't know? I know I wouldn't."

    Can't an external sites.dat file be used with a generic compiled Java app? When I use hleOFXquotes (the Java app for quotes on my site), it remembers the quotes I've previously downloaded, so it must be storing a cookie or ini file, right?

    And you're right - I really don't know if anything fishy is going on with that app, but in this case the data is quotes, not account passwords, etc.

  52. The need for "open source code" is that your account information (including user-name and password) is being used to retrieve your statements, but could just as easily be piped off to a 3rd party server in parallel without your knowledge. If you can't review the source code, then there's no way to know what is really happening... thus the requirement for "open source". If the developer were a recognized company (like Microsoft or Intuit), then this would not be an issue (i.e., we "trust" them).

  53. Regarding the "D" in the comment... that was a typo, but I was using it to debug and didn't think much about it. Not sure why I chose "R"... maybe for review? I think, based on your comment, that adding "stock/fund quotes" as an option under "test accounts" may be useful? (i.e., listing stocks/funds as just another "account" there)

  54. "Stock/fund" testing is now an option under "test account".

  55. Thanks Robert.

    Got a question for you. Looking at my ofx files I see that statement balances are downloading but they aren't showing up in Money. What is your experience now that you have Online updates turned off? I'm assuming that because my accounts still have online updating set, that the statement data isn't altered from my downloaded OFX files. My accounts that aren't "online" show the statement balance after QIF downloads.

  56. All of my statement balances are importing and updating correctly, so I think you are correct regarding your accounts that are still "online" in Money itself. You could test by creating a copy of your Money file, removing the online connections in the copy, and then testing the OFX downloads there.

  57. I've confirmed that once the accounts are offline, then the statement data also imports into Money.

    I also like the test quotes option, because I've been doing some experimenting with quotes for International stocks as well as Options for my Blog.

    Another question; would spaces in the stock symbol throw off the scripts? I'm thinking we could throw in an [options] section into sites.dat that would accept everything until a # (remark) was encountered, unless perhaps the [stocks] section already works that way? Then it's just a matter as to whether would handle the spaces.

    Again, I'm just throwing ideas out, but just say NO! if they seem to involved :)

    And are [stocks] and [funds] really handled seperately?

  58. At present, spaces in the stock symbols should remain intact, but the script won't pass it correctly to Yahoo!. If you could explain how the options quotes actually show up in the Yahoo! URL, I can look to see how to incorporate support. It may be an easy modification (i.e., replace spaces with some other character sequence).

    As for "stocks" and "funds"... I had removed the 'funds' section when testing and it worked fine, but the script specifically includes an additional field for FUNDS named "MFTYPE = OPENEND". I didn't know how Yahoo! uses the field, so I left the FUNDS grouping in-tact. Does Yahoo provide different quotes for "open" vs "close" ended funds, and do they both have the same symbol?

  59. As you see on my Blog post right now it looks like Yahoo! wont be using the spaces per so maybe we won't have to do anything. It's all in limbo until May I guess as they are still using old symbols or transitionary symbols.

    Thanx for the info on stocks vs. funds - I'll look into that some more. By the way, I made the promotion of your scripts a "sticky" so that post doesn't get buried - I really like what you've done thus far :)

  60. It appears that closed end funds are more like stocks and should probably be in that group when requesting quotes. Most mutual funds are open ended though.

    I found this on the web: "You can, but it's probably not worth it. You can force the download to go
    into a savable ofx file. You can then manually edit the ofx file to use the
    coding for a stock instead of a mutual fund. It involves changing 3 or 4
    lines of code (changing to and adding "
    Other". Then you can save the ofx file and import it into Money. Possible,
    but probably more work than the manual entry of the data."

  61. Bobby,

    I've just realized that I've not used your name for your script package "PocketSense", so I've remedied that in my blog. Perhaps you'll link to my blog now that I've published a webpage add-in to your great package that allows users to access your scripts from within the Money program to download their OFX data.

    Try it out and let me know how you like it.

  62. I'll try to take a look at that this weekend Dan. I hope to add another page to the site with links/references/etc, and I'll be sure to put a pointer to your blog there. I haven't taken time to focus on this lately. Too many "pokers in the fire" so to speak.

    I hope others find the code useful, but there haven't been any other comments here... so we may still be the only users!

  63. I don't think you are the only users. This package is a good alternative to continue using MS Money. Many more will be using it in the near future as Money folds. I will be running it in conjunction with Money until it stops working, just so I don't have to start over with another finance package.

    I do have a request, can an option be created so that the OFX file can be created but not submitted to Money. If I open the wrong Money file first, the data goes into my live file instead of my test file. That way I can open the file and look at it then delete it before it is imported. (Did I miss this in the documentation?)

    Thanks for a great package.

  64. I kind-of second that...

    What I've had in mind is making the Review option "not hidden"

    doit = raw_input("Download transactions? (Y/N/R) ")

    it would set the debug flag, and then at the end: Finished. OFX files can be reviewed in the XFER folder /or/ Press Enter to submit files to Money for import.

    This option would enable the user to keep the command shell on top (visible) if using my new web page option as well; right now it only stays on top until the first file imports. Is it possible to use "os.startfile(xfrdir\*.ofx)" to do this?

  65. I like the suggestions for confirming the uploads to Money, and feel it should be "non-intrusive". It could be a Setup option, or an interactive option during download. I'll look into this soon and post an update.

  66. Credit card setup went fine first try.
    Investment account, not as good, but I did find the problem. You do not have a "fid" line in the investment template. After adding it in, that now works fine.
    Bank account, I still cannot get it to work. Not sure where the problem is since I do not get a file in the "xfr" folder. I assume at this point the problem lies within the program and not the bank. (I did get a file in the xfr folder when i was having problems with the investment account, so I should have gotten one under bank if that was the problem.)
    In the meantime, I will keep banging on this.

    Thanks for the solution to MSMoney.


  67. I am getting closer.
    I did manage to get a file in the "xfr" folder that says: "Validation error!"
    The error when testing the account says:
    "Invalid OFX message.
    An error occurred while trying to download the statement.
    Review .\xfr\BANK_OF_AMERICA20100228130620.ofx for possible clues...
    An error occurred while testing the new account.
    Press Enter to continue..."
    I did read a comment in the ofxhome page for bank of america :
    "The server doesn't seem to support MFA, since it dislikes VERSION:103 in the header. VERSION: 102 works."
    Does that help?


  68. Found it!
    Has to do with Bank of America, Appid & version, missing parameter.

    Bank of America requires use of Money NOT Quicken. Need to change Appid to "Money" and AppVer to "1600". There is a parameter missing for bank accounts. Need type of account (CHECKING/SAVING) in addition to account number.

    Still cannot get it to work with this setup, but it does work with TFB script. Just needs a little more work here.


  69. Ray: Lets go through each part of this. First, we need to make sure the site entry is correct. Next, we need to get the APPID and AppVer correct. Last, we need to make sure the type of account is entered correctly (checking/saving).

    The first of these occurs in the sites.dat file, meaning that the site information (including the FID) should have a full entry there. The templates didn't try to include every field, as they're only examples and vary by site, so it's important to get all the right values there from the OFX database or from the institution. I'll review my code mods, but I think we can have all the blank entries there we want without affecting function. If so, I may edit the template to list all fields for all types, leaving blank as appropriate.

    The AppID and AppVer are currently hard-coded, but I think that needs to change. (FWIW, the same is hard-coded in the original scripts and the version posted by TFB.) I can add an option (or site field codes) to define a unique value for sites that require it, but I'm curious why it works with the TFB scripts (which use the Quicken identifiers)?

    Last, the type of account "checking" or "savings" is defined during account setup when running It will only ask for the type if the site entry is defined as AcctType: BASMT. This parameter is not defined in the sites.dat file.

    To help resolve that last part, feel free to post your sites.dat entry for BOA here and we'll sort through it step by step.

  70. Ray: I should have mentioned this before, but I don't personally download transactions from our bank (credit union). If you are willing, I will work with you to get your setup going and, in the process, be helping others too.

    What I'll do is make "beta" modifications based on your feedback and let you test them with your setup. Once we're satisfied, I'll roll it into the live package snapshot.

  71. Robert,
    When I did the setup for BASMT, it did not ask me for checking or saving account type so that could be where it went wrong. As far as appid or appver that could be unique to Bank of America. I did not contact them about changing my account to quicken. Other than that, everything worked fine for my credit cards and investments as it was. I would be more than happy to test any changes. You can contact me at r dot rouillard at att dot net.

  72. OK Robert,

    Got it going doing the following.
    In file
    1. I changed appid to "Money"
    2. Changed appver tp "1600"
    3. Changed line 190 to acct_type = "CHECKING" #account[2]
    It runs good like this, of course that only works for a checking account.
    The variable account[2] is not being set.


  73. The pest again.

    Okay...Line 63 of should read:
    if FieldVal(Sites[sitename],'CAPS')[1] == 'BASTMT' :
    Note the [1] added to the if statement.
    undo change 3 above and that fixes this problem.
    As far as appid and appver, that is up to you. I think it should be a variable, but it's not really needed for most.


  74. I'll fix that bug and I'm also going to make appid and appver as an optional setting in sites.dat. BTW, that bug came from me changing how I was handling site data internally, and I never retested the bank/checking account configuration after I did those changes. Thanks for helping with this!

  75. Robert,

    Ref: "but I think we can have all the blank entries there we want without affecting function. If so, I may edit the template to list all fields for all types, leaving blank as appropriate."

    Back before you used sites.dat, I tried doing the same (blank entries), and got errors. As long as you ignore the blank entries when transferring them to your python script, you should be fine though. And your other Beta tester (me :) ) doesn't have a bank account either (with OFX capabilities).

  76. Ray,

    Thanx for helping figure that out!


    In WIndows, the OFX files can be submitted to Money as a batch, so that would be really sweet if the script could do the same (if you find an easy way to do it :) ), rather than one at a time as the array is processed. Another advantage as Ray aluded to, is that if upon inspection you don't like a certain OFX file, you simply delete it from the folder prior to acknowledging that last prompt to submit the files to Money.

  77. Here's a beta test version to address the appID, appVER and checking/savings bug. In particular, there are new fields in the sites.dat template (sites.template) for AppID and AppVer. I also expanded every entry to include all fields... although I'm not sure if that helps or hurts. Of course, these fields can be added to existing sites.dat files.

    When implementing the parser approach, I had written a wrapper function to get field name and value pairs... which skips blank entries, so anything that is blank will be skipped, as will any field name that isn't recognized.

    Test code:

  78. Robert,

    I gave the beta a try and got this...

    C:\Documents and Settings\Dan's\My Documents\Microsoft Money\Beta>python setup.p
    Traceback (most recent call last):
    File "", line 130, in
    userdat = site_cfg.site_cfg()
    File "C:\Documents and Settings\Dan's\My Documents\Microsoft Money\Beta\site_c", line 60, in __init__
    File "C:\Documents and Settings\Dan's\My Documents\Microsoft Money\Beta\site_c", line 80, in load_cfg


  79. Try again... that little bug was a bit more involved than I had anticipated!

    I also noticed something that I had missed before. I had used the shutil.copy() function to copy the sites.template to sites.dat. I just noticed that the resulting file looks to have Unix formatting. I didn't notice since I don't use Notepad.exe. I changed the copy method so that the initial file now has standard Windows formatting.

    To make the sites.dat file compatible with notepad.exe, simply open in another text editor that reads unix flavor files (notepad++ is good), copy the text into an empty page and re-save to sites.dat. If you don't use notepad.exe, this isn't needed.

  80. Robert,

    Same problem as Dan.

    Basically needs to ignore lines that begin "#". Right now any line containing one of the variable names crashes setup. Does not matter if its a comment or a blank variable. I got past these problems by removing the variable names from comments and giving each mininterval a value. It did get us past the error, but there are potentially other errors in the making.


  81. Thanks Dan and Ray. Try the latest version (same link above). It was little more involved, but mostly centered around the values that are stored as integers. Needed a check for null values on those parameters prior to attempting an int() conversion. I did a few more edits while I was in there... see note above for one.

  82. hi Robert,
    Still testing the second beta version. I started everything from scratch. Credit Cards set up okay. I am still trying to figure out why INVESTMENTS are broken. Appears to be a login problem. Have not gotten to Bank yet. I'll send you an update later tonight or tomorrow morning.

  83. That's it for tonight.
    Investment is now working provided that I use "fiorg" field in sites.dat, but I had to blank out the "brokerid" field. Last version I had the same data in both fields, but this version did not work with it that way.
    Bank no longer works. The error message coming back is bad username/password combo. The appid and appver seem to be working since the bank is not invalidating the request.

  84. Ray: what investment site are you using? I ask, because the scripts first look for a BROKERID field and, if empty, use the FIORG field as the BROKERID for the site connection. Feel free to post your actual site entry here.

    The bank issue is more concerning. I will use your email directly to discuss that issue one on one, and then summarize the fix here.

  85. I'm testing today too. I have a National City account that i'll attempt to set up. PNC bought them but there's one more month of OFX downloads.


  86. Here is the difference between investment working and not working:

    FIORG = Blank ..... BROKERID = Filled in
    Error message returned from software is:

    Site PROFIT does not have a (REQUIRED) BrokerID or FIORG value defined.
    An error occurred while trying to download the statement.
    An error occurred while testing the new account.
    Press Enter to continue...

    FIORG = Filled in ..... BROKERID = Filled in

    Part of header sent:


    FIORG = Filled in ..... BROKERID = Blank

    Works as expected::::::::
    Part of header sent:


    Note the FID line.

    I did get part of bank to work with no changes to your software. The other bank problem is not yours. Bank of America has some really, really strict parameter checking and I have not figured it out yet. Is there a way to trap what Money sends to an institution so I can check it against what you are sending? I am sure it is just one parameter off.


  87. Thanks Ray, I'll sift through the broker/fiorg info you posted and see if anything registers in the code. As for capturing Money traffic... not that I'm aware of, it's an encrypted link.

    I started writing you a message this morning but got sidetracked. Enable the Debug flag in, and the entire OFX header will dump to the screen, the connection will be attempted, and the response recorded in the .\xfr\account_time.ofx file. Nothing will transfer to Money when Debug==True.

  88. Got it!
    Guess what Bank of America wants for account type?
    In I changed line 21 to:
    BankTypes = ['CHECKING', 'SAVINGS', 'MONEYMRKT']
    Now it works. Just took a little more digging around.


  89. OK... so we need "MONEYMRKT" as a bank account type option? Believe it or not, I didn't hard-code the banktype selection since I wondered about whether there may be other bank account types (money market was one). Great!

  90. Now I want to focus on the INVSTMT issue you're having. I sent you an email, but most of that focused on the bank issue. A copy of the site entry for the trouble child may help on this end.

  91. Note to all:

    If you are able to download OFX data from your financial institution’s website, use the Save option rather than Open, and you should be able to find all of your your settings (except url) in that file. This is particularly useful when you aren’t having luck with the other sources. Many banks use the routing number as the the bankid, but for National City bank, I had no luck until I opened up the website induced OFX file and used that bankid data instead.


    Is that how you found MONEYMRKT?

    I have a BOA credit card, but unless you have a checking account with them, they won't give you OFX access except via the website statement download. They keep asking me why I don't use that card much and I told them "because Discover gives me OFX access so I don't have to wait 30 days to input my transactions into Money".


    Success with the latest Beta setting up a bank account. I think you've got the bugs worked out :)


  92. Robert,
    As long as you are adding account types, might as well add the last one:
    CHECKING Checking
    SAVINGS Savings
    MONEYMRKT Money Market
    CREDITLINE Line of credit
    The spec does not indicate any more.

  93. I've added all of the bank account types and have modified the parser routine to (hopefully) correct the issue Ray was having with an investment account. It was a definite bug in the parser, one that could have caused others problems. The snapshot has been updated and I'll wait to hear back from Ray to see how it works for his account.

  94. An interesting observation while looking at Ray's sites.dat file. I think that he had used the minInterval in a way that I hadn't originally planned, but it brought up something that I think may be beneficial. I'm considering adding another field, named "interval", which would directly override the defaultInterval value for a specific site. This value would *always* be used for the site (if the value isn't null of course).

    Any thoughts on that?

  95. I'm just trying to think of where that would be useful. IF you went on vacation for 2 months, you would just change the default. If my BOA worked I guess that value would be 30 days, since they only export statements.

    By the way I experimented (os.system and os.startfile) and could not find a way to submit the OFX files as a batch at the end of the downloads (after acknowledgment). But then, I don't know Python very well like you do.


  96. By the way, here is a suggestion for sites.dat file:

    #--- Open-end funds only; use the Stocks section for closed-end funds ---


  97. Dan: I plan to add the ability to confirm uploads to Money... but wanted to get the bugs out of the the parser issues first. The more I change at once, the greater the number of variations on errors :)

    Adding an overriding "interval" field wouldn't affect using the defaultInterval, as it would be optional. I wouldn't use it either, but could see how someone may want to hard-code an interval for a specific site for some reason specific to how the site handles things?

  98. One more quick thing before I leave for the day.
    Line 216
    doit = raw_input('Remove file encryption and password protectoin (Y/N) ').upper()
    shud be: "protection"
    Not that my spelling is any better!


  99. Updated the beta snapshot with all bug-fixes "to date".

  100. To answer one of my earlier posts, There IS a way to trap what money sends and receives from institutions when doing online transfers.

    Set the following registry key only after money is running:
    Be sure to clear it when done.
    Note: the "16.0" is version dependent.

    Check this link for more info:


  101. Excellent info to have. Thanks Ray.

  102. Wow another version already.
    I have not tested this version yet as I am still trying March 4 version. I did run into one problem and made a change to "Getdata".
    My financial institution is a "Day Late and a Dollar Short". That means that the statement is based on the previous day's closing price and positions. What happens is that the price of funds are the previous close not the current. When quotes get imported, these are ignored because Money has already received a price today and the quote price is ignored. (Wish I could find a way for money to accept quotes more than once a day.) To fix this, I have changed Getdata so that quotes are generated and sent to Money before accounts are processed. So far so good. Unfortunately it takes a long time to test since I can only do this once a day.
    Really need to solve the quotes problem.

  103. Ray: Are you saying that submitting quotes to Money before the account statements more-or-less takes care of your issue? I don't use the quote function for anything important at present, and really don't track my accounts "daily" anyways (that would be stressful). The sequence doesn't matter for any other reason that I'm aware.

    p.s. The new version simply allows stock/fund quotes even if no statements are downloaded... so it doesn't affect you.

  104. Ray,

    I'm trying to understand your post. Normally you wouldn't request quotes for stocks that you have in your investment accounts, because the statement downloads will take care of those quotes for you. I have stocks in DRIP plans that don't offer OFX downloads as an example, and those are the investments that I plug into the quotes section. Don't enter the stock symbols for your broker investments and you should be good, right?


  105. I just tested an investment account that is indeed a day behind, but the quote was entered for the previous day using the previous date, just like the online updates perform, so as long as you don't enter the symbols for those investments in the quotes section, all is well.


  106. I don't think it is stressful. I just do a download of all my investments, credit cards, and bank statements to see what action has taken place. Although rare, I have caught things with my accounts and was able to take care of them quickly.
    As far as broker statements, I have always relied on Money to get current quotes so I know what the investment did today (just something I like to see). Without Money getting the quotes, the prices quotted from the statement are old. By doing quotes before accounts, that seems to fix the problem for now.
    If you do not track your accounts daily, none of this matters. Since I have a tendency to travel a lot, I want to see errors as soon as possible, especially credit cards.

  107. Re: "stressful"... I was just kidding around :) I don't have any preference regarding which method is used to update ticker values (quotes or the statement), so this may be something worth adding as an option in Setup. I could see some users wanting the exact broker-supplied value, and others wanting "today's" value.

  108. Ray,
    ref: By doing quotes before accounts, that seems to fix the problem for now.

    Only problem with that strategy is that the next day when you get your day-old closing quotes, they won't show up in Money since a quote is already there for that date.

    That is another plus for Bobby's planned interactive mode - you'll be able to review the current quotes in the script window and then skip sending those quotes to Money!


  109. Dan,
    It's actually no different than I see now. The past quotes are all available to see except it says "update" instead of "online".
    I run both versions every day and compare account balances and portfolio pages to make sure both are the same and all transactions have been picked up.
    The only thing I do not like about the script method versus the Money On-Line method is that the on-line gives me a change column value in the portfolio view, while the script method always gives me "unch". That is why I keep looking for a way to upload quotes to Money so it thinks it is on-line. There has to be a way to fake it out, just have not found it yet.

  110. Data won't upload to MS Money
    Hi Everyone,
    I just started using PocketSense.
    Win XP SP3, MS Money Plus Deluxe (updates expired), Pocket Sense ver. 18-MAR-2010. w/encryption.
    I disabled Online Updates in MS Money for all accounts prior to using PS.
    So far I've been able to successfully
    download .ofx files from Fidelity, Ameritrade, Chase VISA, and Regions Bank. (Regions Bank was tricky)
    The .ofx files d/l to:
    E:\My Documents\PocketSense\xfr
    I run in [I] mode and it DOES ask: Upload online data to Money? It does NOT, however, seem to be sending the txns. to Money.
    I CAN manually import the .ofx files thru MS Money using File > Import > Downloaded Statement
    After getting kicked to the curb by MS, your program offers an opportunity to score one for the little guy. Great job! I hope someone here can help me with my little glitch.
    -Kevin N.

  111. See if this helps...

    1.Go to control panel->internet options
    2.Temporary Internet files->delete files->delete all offline content
    3.Temporary Internet files->settings->reduce the cache size to a smaller figure, say 10-20MB
    4.In addition, it is worth changing (if set): Advanced->do not save encrypted pages to disk.
    5.Try the OFX statement download again
    The above procedure seems to force Internet Explorer to handle the downloaded file.

    More here...

    It sounds like an issue with Money's Import Handler - reinstalling Money might also fix.

  112. Hi Dan,
    Thanks for your reply.

    I probably also should've mentioned that I'm using IE 8.

    I tried the suggestions. It still appears that Money is NOT getting the txns

    I may wait a day or two to make certain that I actually have new txns. and then try at it again.

    I'm reticent to uninstall / re-install Money as I'm not sure if the 'activation' process will kick in on the re-install. If that happens I'm done for.

    If you can think of anything else to try, I would appreciate you passing them along.

    Thanks again.

    -Kevin N.

  113. Kevin,

    The script "directly executes" ofx files, similar to double-clicking the file in Windows File Explorer. Open a file explorer window, and go to Tools->Folder Options->File Types. Look in the "registered file types" for the OFX file extension. It should be registered to "Microsoft Money Import Handler", or something similar. If not, then that's the culprit. Check this and report what you find.

    On my system (also Money Plus Deluxe), the import handler is in a hidden "system" directory in the Money program path. On my system, it is the following file that handles imports:

    C:\Program Files\Microsoft Money Plus\MNYCoreFiles\mnyimprt.exe

    The .\MNYCoreFiles directory is a HIDDEN SYSTEM folder, so you can only see it by enabling view of it's view from the Tools->Folder Options->View menu in a File Explorer window. You should be able to manually define this as the "handler" for OFX file types from the Tools menu mentioned above, or reinstalling Money.

  114. Hi Robert,

    Thank you for your reply.

    Just a brief back history. I was using Moneydance along side MS Money for about a year now and through no fault of Moneydance, .ofx files were associated to 'Moneydance data files'. I say through no fault of Moneydance because during installation of MD the user IS asked whether or not to associate .qif, .ofx, and .qfx files with MD.

    I had a feeling that the .ofx file association was the culprit. I had tried changing it back to 'MS Money'. This just changed the file association to "OFX file" but had no other effect as far as getting the txns into Money.

    As for associating .ofx files with mnyimprt.exe, I surprisingly, had no luck with getting that to work either.

    As my final do-or-die attempt, I did an in place re-install of Money and THAT did the trick. I can now run and the tnxs. are sent to Money as expected.

    .ofx files are now associated to 'Open Financial Exchage files' with the Money icon.

    As an added bonus, Moneydance is NOT effected by the .ofx association change.

    Thank you for a great program and for your help with this issue.

    -Kevin N.

  115. Hi Bobby,

    Me again!

    I've been banging on this for a month now, and it looks really good. The only problem I am having is getting daily quotes into money before the broker statement which means the quotes are a day old. Yes,I do have the flag set for quotes first in "sites.dat", but when I look at file times in the "xfr" directory, quotes are later than the other ofx files. Am I missing something? I would expect the quotes.ofx file to enter the data into money and the broker file to follow with the rest of the data. It does work correctly when I send them in manually ( not sequential like getdata does ).
    Is it possible that msft is not processing the files in order when they hit the transfer directory?
    I will keep looking into this to see what I can find.


  116. Kevin: Great to hear. Have fun.

    Ray: Not sure. If the flag is set for quotes to upload first, then this has no affect on the order they are downloaded. Quotes are always "grabbed from Yahoo" last. The flag tells the script to send the quotes to Money first. If that's not happening, then I'm confused myself. if you run in interactive mode and verify each statement upload to Money, you should be able to see whether quotes are going first.

    A different issue that has been noted is that the Quotes routine does not currently use the actual quote date, but instead uses the date that the script is executed plus a day. I didn't write that, but it's how it currently works. On my "to do list" is to rewrite the quotes script to use the actual quote date. There is a brief discussion on the 2nd thread of comments:

    I haven't had time recently to work on this, but I'm going to sit down soon and try to wrap in some of the comment and issues hat have been noted.

  117. Bobby,
    I think your code is okay. If the flag is set, quotes do download first (verified in interactive mode). I think the problem may be how Money/Microsoft processes the data sent to "os.startfile". If the files come too quickly they may not be processed in order sent. What confused me was the fact that it did not happen every time. It's difficult to test something that only occurs once per day.

  118. If it's a time issue, I can add a small delay between each. If you do it interactively a few times and never have an issue, then that would verify that "first in isn't first out" as the culprit. If you could do that, it would help (I don't re-collect quotes that are statements, so I don't see this on my system). Thank you.

  119. Evidently your downloads do not mimic what msmoney does. Even though statements are accepted by money, it always takes it current quote from its server, NOT the statement. The data in the statement is used for individual records Not the latest quote (unless there is no quote from the MSN server).
    I can confirm that you are putting out the files in the proper order, but money is taking them in and processing them in its own order (not sure what that is right now). I can also see this as being a problem when files are created and sent to money, but money is not running at the time. When it starts the files are processed in some? order.
    My next test will be to change the script so that the quotes file will be named aaaquotes.ofx and see if its an alpha thing. If that does not work, I will try to have the files created in a different time order.
    I'll let you know what I find.

  120. Hello Robert,
    It's not a filename thing.
    It has to do with the date/time the file was generated plus how long it takes money to import it among other things. Best I can tell right now:

    If you send and import files one at a time, the order of import is preserved.
    If sent in rapid order, it does not follow the order the files were sent in. If two or more files are seen by money, it goes by chronological order, oldest first.
    If all files are sent in, then money is started, its by chronological order. Oldest first.
    I will not be able to properly test this again until Monday when these conditions arise again.
    I cannot really call it a bug, as much as money doing what it thinks is right. If there are several files ready for money, then it should pick the oldest first since it wants to do transactions in order. Money may not care about the filename as much as the file content, but it cannot risk getting the transactions out of sequence.

  121. Thanks Ray. Very helpful! This is an easy fix, but I'll wait for your response confirming that "files are imported by date of creation". I'm making a "to do" list for the next revision, and this will be included. The other big one will be to try to use actual quote date/time rather than the current method of assigning all quotes to an offset from today's date.

  122. Testing More Than 9 Accounts.

    Hi Robert,

    In an earlier post dated March 07,2010 microsoftmoneyoffline asked about adding beyond the 8 accounts that he was working with.

    I've created 11 accounts in

    When I choose option 7. 'Test Account', I'm presented with a list of the accounts 1~11, 12 is 'Stock/Fund Prices' and the next line is 0. 'None'.

    I can successfully 'test' accounts 1~9. I can NOT however 'test' accounts 10 or 11.

    When I input either 10 or 11 and press Enter, the 'Test Account #: [0]' prompt is repeated.

    12 'Stock/Fund Price' CAN be 'tested' and 0 'None' functions as expected. downloads ALL 11 accounts and Stock/Fund Prices as expected.

    I hope this information helps.

    -Kevin N.

  123. Kevin: Thank you for the feedback on that. I'll add this as a bug fix for the next release.

  124. Comments closed on this post. See for the continuation.