Friday, March 24, 2017

New version released

Updated:  10-Apr-2017

Spurred by recent OFX server changes at Discover (and apparently others), the script package has been updated, and is posted for download @ https://sites.google.com/site/pocketsense/home/msmoneyfixp1/p2.  Thanks to everyone for feedback and help!  I think we're in good shape for a while, but it would be really nice if the bank developers would abide by the OFX standard ;)

Summary of Changes

  • Updated url requests to work with recent changes at Discover and others.
  • Added support for OFX 2.x (tested with ofxVer=211 @ Discover, but they support 102 and 103 too).
  • Added support for user-specific clientUID pairs (by site url + username).
  • Added skipZeroTransactions option to sites.dat configuration.  If Yes, remove $0.00 transactions from downloaded statements.
  • Minor code updates and bug patches.

33 comments:

  1. Hi Robert. With the end of the month I was finally able to test the sign fix for Net Benefits in the new version. It turns out it doesn't work, but I've spotted the reason why.

    INVBUY and INVSELL transactions have the following order

    REINVEST has a different order (of course, why not, why have a standard?):

    As a result the regex that's used in _scrubINVsign does not pick up REINVEST transactions

    Thanks
    Carlos

    ReplyDelete
    Replies
    1. Thanks Carlos. I think something's missing from the post above?

      Delete
    2. Ugh. I forgot tags get interpreted by the comment engine.
      INVBUY and INVSELL transactions have the following order: UNITS, UNITPRICE, TOTAL
      REINVEST order is: TOTAL, UNITS, UNITPRICE

      I can provide you with an example, if you want. It's a trivial change, but a different scrubber is needed for REINVEST than for INVBUY/INVSELL.

      Delete
    3. Gotcha. Pretty sure I know what's needed, but since I can't test, it would be quickest if you send me the scrub routine you're using for REINVEST transactions, and I'll add it. Send to pocketsense1/at/gmail/dot/com.

      Delete
  2. Thank you so much for your work on this Robert!

    -ameridan

    ReplyDelete
  3. New version posted, including scrubber update provided by Carlos. Specifically, malformed REINVEST transactions should now be handled correctly.

    ReplyDelete
  4. Um, _scrubREINVESTsign is not actually called anywhere, is it?

    ReplyDelete
    Replies
    1. Well... that's embarrassing. Fixed.

      Delete
  5. Yeah Robert, thanks for maintaining this site. This has been a great tool to keep MSMoney alive.
    So, I was wondering if it would be a challenge to add a feature in SETUP to re-arrange the Site list order of execution. Thanks again!

    ReplyDelete
    Replies
    1. I've been tied up on other challenges, so a bit behind here. When you say "rearrange", do you mean a sort order, or a "user decided" order? I'm also wondering "why"?

      Delete
  6. I'm trying to set up MSMoney and I've come accross a couple of problems. The first is when I try to run the Setup.py file ,nothing happens. I'm running Windows 10 - 64bit. The second is the link (http://www.ofxhome.com/index.php/home/directory) times out and the saved copy of this site does not allow to look in the directory, using the other option (http://moneydance.com/synch/moneydance/fi2004.dict) I can only find banks that are based in the U.S.

    ReplyDelete
    Replies
    1. Not sure about ofxhome, etc., but curious if you got the scripts working? If nothing happens via Setup.py, then that's likely a problem w/ the Python install. Generally, ActiveState Python configures everything correctly in Windows, but some do not.

      Delete
  7. Found this while looking for a fix to my problem (https://pypi.python.org/pypi/csb43/0.3.2),
    Tools for converting from the Spanish banks’ format CSB norm 43 (CSB43),
    (norma 43 del Consejo Superior Bancario [CSB43] / Asociación Española de Banca [AEB43]),
    Supported formats:
    OFX XML
    HomeBank CSV
    HTML (python2)
    JSON
    ODS: OpenDocument spreadsheet (python2)
    CSV, TSV: comma- or tab- separated values (python2)
    XLS: Microsoft Excel spreadsheet (python2)
    XLSX: OOXML spreadsheet (python2)
    YAML
    might come in handy for others.

    ReplyDelete
  8. Anybody else getting Error code 2003 when attempting Discover Card downloads? I'm hoping it's temporary, but 3 days now...

    ReplyDelete
  9. I thought I might be one beta behind, so I just downloaded the latest version of Pocketsense, retried Discover, and now I get "Access Denied; You don't have permission to access "http://ofx.discovercard.com/" on this server."

    ReplyDelete
  10. Yes I'm getting same access denied error with discover again [sigh]. Was working fine with Beta for many weeks, then stopped working so I decided to upgrade to latest release but alas it still didn't work. Another discover problem I suspect.

    ReplyDelete
  11. I tested just now, and have the same result. Will look into it...

    ReplyDelete
  12. I downloaded a recent Discover Card statement (qfx format) using the web portal, to see if anything had changed. The account number is different, having a format of "Discovr60xxxxxxxxx####", where the #### looks like part of the original account, and the xxx... is something unique. I added a new account in Setup using this account# (exactly), with the same user/password, and tested w/ success.

    I repeated the above for Discover Bank, since I use them as well, and found the same type of change. It too worked with the (new) encoded account#.

    Be sure to test during Setup, and send the statement to Money and re-match it to the existing account @ import. Since it has a different acct#, it must be re-matched when prompted.

    I noticed the longer acct# doesn't fit in the existing Setup list output, so I'll widen the account column.

    ReplyDelete
  13. Great job Robert! That worked for me too.

    ReplyDelete
  14. Robert you are a genius! Works great again. Mine was in the format DiscovrXXXXXXXXXXX#### where #### was the last 4 digits. I did not have the 60 to start. But that random number worked. Must be a security thing for Discover that they just failed to tell anyone or put on their help pages. Thanks Again!

    ReplyDelete
  15. This is so weird.... I have a few Discover Bank accounts, and each one has a different value for "XXXXXXXXXXX"... I had to download each one using the web site and look at the file!

    ReplyDelete
  16. Updated to the May 11 version, but I'm still getting Discover errors. Any suggestions?
    OFX file shows:

    You don't have permission to access "http://ofx.discovercard.com/" on this server.
    Reference #18.c9ee3e17.1494689640.146a2b8

    ReplyDelete
    Replies
    1. See post above (5/12/17 @ 727a)

      Delete
    2. Hi Robert,
      thanks for the quick response!
      Now I understand... instead of using my discover card account number, I need to download a qfx file from the website and locate the account number in that file.

      Delete
  17. Python question: I copied a *.mny file from Win7 to a different PC with Win10 and ever since the "get data" does not work on the Win10 PC, Pop up says: python.exe stopped working Windows will close the program. Would it help to reinstall Python 2.7 and if so, would the new program install itself onto the old or should I uninstall the old Python first?

    ReplyDelete
  18. So weird. Thanks for discovering the Discover fix Robert.

    Also, I expect this as come up before but does anyone else get a bunch of zero amount transactions when downloading from Discover? I get so many of them (they only have payees but no memos or amounts). It'd be interesting if there's a way to stop those from being included in the ofx file.

    ReplyDelete
    Replies
    1. The newer versions of the script's sites.dat file has a 'skipZeroTransactions' Yes/No setting.

      Delete
  19. Robert,

    Thanks for this great work. Is there any chance that you have source control, or history of your scripts somewhere? I'm also using a script that appears to have been originally derived from the same source, available as the "ofxclient" package on PyPI or at https://github.com/captin411/ofxclient

    I was using it to download from Discover, and it broke last week. I was hoping to find a way to merge your OFX 2.0 work into it.

    Thanks,
    Jason Antman

    ReplyDelete
    Replies
    1. Sorry Jason... but no source control. Probably should have, but never did.

      Delete
    2. I have the following downloaded zips of pocketsense:

      ofxpy_pocketsense_beta-20130217.zip
      ofxpy_pocketsense_beta-20170307.zip
      ofxpy_pocketsense_beta-20170312.zip
      ofxpy_pocketsense_beta-20170315.zip
      ofxpy_pocketsense-20130714.zip
      ofxpy_pocketsense-20140906.zip
      ofxpy_pocketsense-20170324.zip
      ofxpy_pocketsense-20170410.zip

      plus a ofxpy_pocketsense_beta.zip where the latest file (rlib.py) is dated 2012-08-29.

      Each individual archive is about 40kb. I'm sure if I've missed any, someone else will have a copy. Robert may already have all of these.

      From these versions, it should be fairly easy to initiate source control - just start with the oldest and check in the changes in order.

      Delete
  20. Chase and Discover down this morning. Hoping it is temporary. Chase gives empty OFX files (0 bytes). Discover gives ERROR OFX status.

    ReplyDelete
    Replies
    1. I guess I posted too quickly before reading the thread. Downloaded file directly from web site, and like Harold and Robert said, it has a random account number. Changed the account number to the new random number, and all is well. Let's hope they don't start changing it every 5 minutes.

      Delete