Blogger Ron said... Cal (or Robert or other knowledgeable Python programmer)
Taking your and NxTk's advice, I wrote a scrubber that changes the last digit of the FITID to a "1".
It seems to work and be restricted to only my savings account at TD Bank.
However, this is the first time I've programmed in Python. Would you mind taking a look and seeing if I did something that might cause a problem down the road?
``` #TD Bank: Remove last digit of FITID from Savings Transactions #29 Jan 2024
from control2 import * from scrubber import scrubPrint import re
def scrub(ofx, siteURL='', accType=''):
if 'eftxweb/td.ofx' in siteURL.lower() and accType=='BASTMT': global stat stat=False
#Check that we are dealing with the correct account if re.search("ACCTID>##############", ofx): scrubPrint('TD Bank Matched Savings Account')
#We are, so change the last digit to a "1" ofx = re.sub(r"(?PFITID>\d{19})\d", r"\g1", ofx) scrubPrint('Last FITID Digit changed to "1"')
I don't follow ofx = re.sub(r"(?PFITID>\d{19})\d", r"\g1", ofx)
I don't know r"\g1". I am not familiar with a \g metacharacter.
And whatever you had in your code, I suspect you has a > character that you did not replace with these four characters: >
Does your code work, or have you not tested it yet? The string I posted in post 200 of http://pocketsense.blogspot.com/2023/07/new-version-available.html should work.
Also, have you checked that all non-duplicated FITIDs end in a "1"? That still might work if the rest of the characters are unique.
The match string is using a Named matching group (FIT). In the replace string the \g says to use everything that was matched by FIT.
For numbered matching groups, one could use \g1, but if one wants to add a 1, the construct becomes \g11 which is confusing to the regex engine because it starts looking for group 11. So I used a named group to avoid that problem.
Does seem to work.
In the savings account, there have been NO non-unique entries. In the checking account, (which this scrubber does not process as I check for the account number before running it), the last digit seems to be anything. I don't have enough data to be sure that the first 19 digits will always uniquely identify any transaction.
And also, at least so far, those non-unique entries have all had the first 19 digits the same, and they have all started with `1`. But I only have a database of two entries -- one deposit transaction and one interest transaction. That'll be a good thing to look for going forward. And should be easy to detect if there is an anomaly. Thanks for pointing that out.
dammit... I meant to write And whatever you had in your code, I suspect you has a < character that you did not replace with these four characters: <
If you want to know how to use pdb to test little code snippets with pdb, let us know. But first tell us that you are able to copy and paste from/to a command window. You can Google how to do that that.
In an earlier post I showed how I tested my string in pdb. The "(Pdb)" is a prompt from pdb -- Python debugger.
NxtTek: Looking at the code as posted here more closely, it seems it did not post as it really exists. It shouldn't run at all unless it matches the bank and account number (and it doesn't here). However, the code as posted is missing indents under both the first and the second nested IF statement. I wonder if that could be your problem. I'll see if I can get it to post correctly.
NxtTek: I couldn't figure out how to post it here properly formatted. Here's a link to a text file containing the code which should be properly formatted: https://1drv.ms/t/s!Aml6o6HIc9uOy28nDi-NrRdzURAI?e=tuaOMJ
It's been mentioned before, but it does look like it is only a matter of time before Fidelity access closes. I found the following text on their web site: "To enhance the protection of your account data, Fidelity has established a secure connection that better controls how third-party websites and apps that you've authorized, and the data aggregators they use, connect to your accounts. Fidelity is requiring these data aggregators to transition to this secure connection. Fidelity users of some third-party websites and apps may experience a disruption in the link between those websites and apps and their Fidelity accounts. "
I'm going to be optimistic, thinking that's in reference to web scraping ;) Fidelity saved the day from Schwab killing their ofx server, but I don't think there are any more saviors out there, if Fidelity does the same. - ameridan
The current situation lets your program (PocketSense in our case) access your transactions and positions directly without involving Yodlee, Quicken etc handling your data. Their announced intention is to require you to go via a third party that they work with to then provide the data to your computer or to a cloud app.
See the official response in https://www.reddit.com/r/fidelityinvestments/comments/17tpgfr/why_doesnt_fidelity_support_ofx_downloads_anymore/?rdt=47495
So it would be more precise and robust than screen scraping, but would still require that your info go to a third party. It would be safer in that the third party would not have your Fidelity credentials. It also would provide an income stream to Fidelity. I don't begrudge them that, but I like to minimize the sharing of my financial data.
Thus the urgency to be able to generate OFX files, that can feed into PocketSense, from the history and positions CSV files that you can download from the Fidelity website. This is extra work over how PocketSense now works with Fidelity, but worth it IMO. I don't know if Vanguard will continue to offer OFX directly,
A free Fidelity to OFX converter from Cal Learner (fidcsv3 or F2O name subject to change):
Ameridan's blog nicely documents where this program/project stands and explains the methodology.
See https://microsoftmoneyoffline.wordpress.com/2024/02/03/cals-fidelity-specific-csv-to-ofx-converter/
While the current main program is named fidcsv3.py, it will probably be renamed to something like F2O. py or f20.py at some point. Since this runs on windows, the filename will not be case-sensitive.
Advantages:
o open source o developed on Python 2.7 as PocketSense uses free o can be used with PocketSense or without (I do recommend PocketSense) o puts both transactions and positions and price info into the OFX files, just as with the OFX files that currently come in from Fidelity via PocketSense o still being improved, so any bugs bug reports or suggestions posted here will be reviewed and probably acted upon o the program finds your latest Fidelity positions and history CSV files in your Downloads folder without further interaction needed.
Disadvantages:
o not professionally coded. o lots of global variables, o not modular. o the coding style evolved since being started March 2022, so method of doing the same thing is not constant thru the program(s) o the code is not already openly published, but that will change if there is any interest. human interface not polished, but will probably improve o user must download the positions and history CSV files from the website for each run, but the program does find those in your Downloads folder without further interaction needed. Download from web is easy enough.
Questions and comments are welcome. I expect interest to pick up after the Fidelity OFX server disappears. But by that time, it would be too late for some things that you would miss out on that would would have made the operation initially easier.
I have decided to open the Fidelity CSV to OFX (F2O) beta to all now. I have a folder that should be accessible given the link. This is a folder rather than a file: https://drive.google.com/drive/folders/1-4fBbRxaPhNYZK0gNHuZ3n-Bdoncy2aR?usp=sharing
In the folder is f2o_beta3.zip which has several files. One file is F2Obeta.doc in Word 97 format, and F2Obeta.txt which is is the text-only version.
Feel free to post comments, bugs, or why did you do something the way you did. The code evolved starting in March 2023.
That worked. I have downloaded the zip file and extracted it but I didn't find any python files and the .doc file seems to be a binary file of some sort???
First, yes I did miss important files. The .doc file did work for me when I just tested. It is intended to be opened by Microsoft Word or other program that can read Word .doc files. I have added a *.txt version too.
New method. This is a folder rather than a file: https://drive.google.com/drive/folders/1-4fBbRxaPhNYZK0gNHuZ3n-Bdoncy2aR?usp=sharing
I will delete the post with the old link. The new method should mean that I don't have to create a new link to change content.
It contains f2o_beta2p.zip and a simple txt file that does nothing.
That worked. I'll try out the F2O_DBgather.py program along with the bat files. The doc file will open in ms word. On my system the .doc extension automatically opens in a text editor. Thanks
The F2O_DBgather.py program worked on my data except for one ofx file (one generated from a cash management account which is just a checking account from Fidelity). It errored because the split_ofx list has only 1 entry in it. ( the ofx does not have any since it only contains a cash entry). Since the file F2O.py was not in the files shared with me, I can't proceed any further.
I will need to deal with that. Currently I have all of my cash in my cash management account used to purchase SPRXX. I should sell some of my SPRXX to generate an entry for the core position for testing. I had intended for that F2O.py file to be there. It is there now with the same zip filename, but a little bigger.
Both programs (F20.py and F20_DBGather.py) worked well with my data other than the cash management account mentioned previously. Thanks, I think these will be useful in the future.
dealt with it. f2o_beta3p.zip contains a fixed version of F2O_DBgather.py It also has a copy of F2O_DBgather.bat.bk1 which is my zip file, but is intended to be modified for your file structure. It is not called F2O_DBgather.bat in the zip file to avoid accidentally overwriting a batch file you already modified.
Besides the bug fix, F2O_DBgather.py accepts an optional second parameter (integer) that would set the DEBUG level. Default is zero.
I have installed Anaconda, downloaded the scripts, and not having any luck getting started. When I double-click Setup.py, nothing happens. Next, I ran the Anaconda Prompt and typed in Setup.py. This gave me a couple of errors, one on import urllib2 and another on pyDes.py (print 'Invalid Password' - SyntaxError: missing parentheses). Any suggestions on how to get started? Thank you.
In setup.py, doing the "2. List Accounts" choice, I do have ConnectKey: listed for each of two accounts. I did not do anything to set up that. When I do the rare move of my PocketSense to another folder, I copy my connect.key and ofx_config.cfg files over to the new folder.
While I have a Citi bank account, I don't think they provide an OFX server for that. I use their web interface to get a qfx file. I do that very rarely because it is a bit tedious. I make sure the dates I download do not overlap with the transactions in Money because they don't/didn't have the thing about having the FITID the same each time working.
Thank you gents. My sites.dat setup matches yours Cal (and yes, i should have specified that this is indeed for a credit card which has worked beautifully up until now). It appears to be a credential issue, however. I do get a response from the ofx server with the following:
ERROR Information does not match with our record. Please make sure you select the correct Financial Institution and enter correct username/password.
I decided to nuke the connection and set it up again...and yeah, it chokes on my credentials (the same ones that work perfectly when logging in on the citi website). I am a bit stumped on this one if others are not seeing the same... oh well... I do have other ways of getting my citi transactions so it is not a big deal. But this means my pocketsense direct connections are down to just Fidelity and Amex... :S
Unless something has changed, I believe you need to enable direc access on the Citi site. Not at computer now, and don’t recall the details, but you add a new connection from an account menu and *then* add the account in PS during the ~ 10 min window. -Robert
Thanks Robert; I suspected that may be the case, but I looked high and low on the citi website and could not find such a feature... No rush, but if you do find that menu option on your end please do report back. i did find some references to Quicken, but that is all in context of Web connect which of course is different.
Oooo... Now that you say that, I kinda remember that discussion.
In http://pocketsense.blogspot.com/2019/08/comments.html see "After making the sites.dat changes, I did not need to use the Setup.py script, but I did need to use the Citi 10-minute process to relink the PocketSense scripts to my Citi account. The easiest way to do that is to log in to https://citi.com/datamanagement. "
Thank you Cal; that did the trick and I am reconnected through Pocketsense now. Maybe my authorization (prior one evidently from 2022) expired, so i wouldn't be surprised if others find that the connection stops working at some point. How one can find that critical page without knowing the direct link is still a mystery to me. Will make sure to save the link for posterity. Thanks again!
The actual procedure has been on my settings page all along (for posterity) @ https://microsoftmoneyoffline.wordpress.com/2010/10/06/cal-learners-review-fidelity-401k-citi-card-and-vanguard-account-info/
Anyone use the PocketSense scripts to download Regions Bank account data? I've used them for years without issue. My downloads stopped working this past weekend and are still not working. It would appear to me that Regions as shut down their OFX server much like USAA did earlier this year. Just trying to verify that that's the case.
Hi John, Here in Georgia, using the PS scripts, Regions Bank is working as expected with both BASTMT and CCSTMT type accounts. In sites.dat I set userAgent: none Also delay: 3 I set the delay because I was getting bounced out midway through the downloads. HTH -Kevin N.
I started getting an internal server error this past week. I don't receive an OFX response back when I try to download transactions or when I try to set up a new account. It's really strange.
Hi John, Check to make sure your Profile at Regions has not changed. Log on to Regions. Hover over 'Customer Service', select 'Settings'. Click the 'Manage Profiles' link. Here, make sure the options for 'Online' and 'Quickbooks' are checked. Click 'Save'.
Kevin - None of my profile info has changed. I tried doing a completely clean install on another machine to see if I could run the setup script to download just my Regions accounts. The script fails on that machine as well. I suppose Regions could have blacklisted my IP address as one of my OFX account requests for a new account that was setup in January was routinely failing. If they had done that though, I shouldn't be able to log into regions.com either.
I added kevin's recommended changes to my settings page. thx https://microsoftmoneyoffline.wordpress.com/2010/10/06/cal-learners-review-fidelity-401k-citi-card-and-vanguard-account-info/
Hi Dan, Thank you for maintaining your site. It is a trove of valuable information. Just to clarify, my suggested 'delay' setting for Regions Bank and Regions Credit Card is 3 seconds. It was inadvertently entered as 30 seconds on your site.
F2O, a Fidelity-specific CSV to OFX converter project is actively in progress. See https://microsoftmoneyoffline.wordpress.com/2024/02/03/cals-fidelity-specific-csv-to-ofx-converter/
The user could select to have the generated Fid*.ofx files go to the PocketSense import folder, but that would best not be done until the Fidelity OFX server shuts down.
PocketSense would still consolidate the OFX files I think, but I don't think the scrubbers get run on the import folder files.
Regarding a csv converter integrated w/ the existing ofx scripts, I don't currently have such plans. I did write a simple conversion script for personal use last year, but it's limited to banks & credit cards and doesn't currently have a "configuration interface". More precisely, I wrote & use it only for Discover.
More broadly, I see "csv to ofx" as a different animal than "fetching statements via ofx services". It's easier than ofx in many ways for banks/credit but seems tedious for investments, since there's no "standard" for how a firm may report transactions and/or positions. The F2O mentioned above sounds like Fidelity is providing a normalized interface to work with, but I'm not a Fidelity customer so haven't tested.
That's a long-winded answer vs a simple "no", but my thoughts at the moment :|
Is anyone having a problem with downloading transactions from Capital One credit card? I've never been able to use Pocketsense with this account but until today I have been able to manually download them and import the OFX file into Money. Now I'm getting a message from Money about an Invalid format. When I look at the file with Notepad I see this strange sequence of messages:-
I am glad they fixed it. It was clear from that nullnullnullnullnull that the OFX file was not right.
FYI, if you put < symbols into posts, the blogspot software thinks those are the start of tags, and removes them up to the following > symbol.
If you replace all < symbols with these 4 characters < then it should post properly. I think it is not necessary to replace > symbols with > as long as this sentence makes sense (it says "replace > symbols).
I couldn't reply to your original post as comments there are no longer permitted.
I wanted to thank you for your work on porting Pocketsense the Python 3. https://github.com/magnuspyke/ofxscripts
Due to Microsoft ending support for Win 10, I figured that I'd give Linux Mint a try.
After a few newbie failures, I was able to get Pocketsense working as expected in Mint. Failures = not running Setup.py in Terminal, typing Python3 instead of python3 (cAsE sEnSiTiVe) and typing setup.py instead of Setup.py (cAsE sEnSiTiVe)
I know that I won't be able to run MS Money there, but I can and do run Moneydance there.
Blogger Ron said...
ReplyDeleteCal (or Robert or other knowledgeable Python programmer)
Taking your and NxTk's advice, I wrote a scrubber that changes the last digit of the FITID to a "1".
It seems to work and be restricted to only my savings account at TD Bank.
However, this is the first time I've programmed in Python. Would you mind taking a look and seeing if I did something that might cause a problem down the road?
```
#TD Bank: Remove last digit of FITID from Savings Transactions
#29 Jan 2024
from control2 import *
from scrubber import scrubPrint
import re
def scrub(ofx, siteURL='', accType=''):
if 'eftxweb/td.ofx' in siteURL.lower() and accType=='BASTMT':
global stat
stat=False
#Check that we are dealing with the correct account
if re.search("ACCTID>##############", ofx):
scrubPrint('TD Bank Matched Savings Account')
#We are, so change the last digit to a "1"
ofx = re.sub(r"(?PFITID>\d{19})\d", r"\g1", ofx)
scrubPrint('Last FITID Digit changed to "1"')
return ofx
```
I like it! These scrubbers are nice to save as a library, even if you don't have an account, for when an occasion arises.
ReplyDeleteI don't follow
ReplyDeleteofx = re.sub(r"(?PFITID>\d{19})\d", r"\g1", ofx)
I don't know r"\g1". I am not familiar with a \g metacharacter.
And whatever you had in your code, I suspect you has a > character that you did not replace with these four characters: >
Does your code work, or have you not tested it yet? The string I posted in post 200 of http://pocketsense.blogspot.com/2023/07/new-version-available.html should work.
Also, have you checked that all non-duplicated FITIDs end in a "1"? That still might work if the rest of the characters are unique.
Cal Learner.
Cal,
DeleteYes, your correct about those missing brackets.
And yes, I tried the code and it does seem to work
let me repost the regex
re.sub(r"(?P<FIT>FITID>\d{19})\d", r"\g<FIT>1", ofx)
The match string is using a Named matching group (FIT).
In the replace string the \g says to use everything that was matched by FIT.
For numbered matching groups, one could use \g1, but if one wants to add a 1, the construct becomes \g11 which is confusing to the regex engine because it starts looking for group 11. So I used a named group to avoid that problem.
Does seem to work.
In the savings account, there have been NO non-unique entries.
In the checking account, (which this scrubber does not process as I check for the account number before running it), the last digit seems to be anything. I don't have enough data to be sure that the first 19 digits will always uniquely identify any transaction.
For the savings account, that should read there have been NO unique entries.
DeleteAnd also, at least so far, those non-unique entries have all had the first 19 digits the same, and they have all started with `1`. But I only have a database of two entries -- one deposit transaction and one interest transaction. That'll be a good thing to look for going forward. And should be easy to detect if there is an anomaly. Thanks for pointing that out.
Deletedammit... I meant to write And whatever you had in your code, I suspect you has a < character that you did not replace with these four characters: <
ReplyDeleteIf you want to know how to use pdb to test little code snippets with pdb, let us know. But first tell us that you are able to copy and paste from/to a command window. You can Google how to do that that.
In an earlier post I showed how I tested my string in pdb. The "(Pdb)" is a prompt from pdb -- Python debugger.
Cal Learner.
This comment has been removed by the author.
ReplyDeleteWell that scrubber messes up all of my other downloads, so I guess the tdbank filter isn't kosher.
ReplyDeleteNxtTek: Looking at the code as posted here more closely, it seems it did not post as it really exists. It shouldn't run at all unless it matches the bank and account number (and it doesn't here). However, the code as posted is missing indents under both the first and the second nested IF statement. I wonder if that could be your problem. I'll see if I can get it to post correctly.
DeleteNxtTek: I couldn't figure out how to post it here properly formatted. Here's a link to a text file containing the code which should be properly formatted: https://1drv.ms/t/s!Aml6o6HIc9uOy28nDi-NrRdzURAI?e=tuaOMJ
DeleteThanks Ron. that works.
DeleteIt's been mentioned before, but it does look like it is only a matter of time before Fidelity access closes. I found the following text on their web site:
ReplyDelete"To enhance the protection of your account data, Fidelity has established a secure connection that better controls how third-party websites and apps that you've authorized, and the data aggregators they use, connect to your accounts. Fidelity is requiring these data aggregators to transition to this secure connection. Fidelity users of some third-party websites and apps may experience a disruption in the link between those websites and apps and their Fidelity accounts. "
I'm going to be optimistic, thinking that's in reference to web scraping ;)
DeleteFidelity saved the day from Schwab killing their ofx server, but I don't think there are any more saviors out there, if Fidelity does the same.
- ameridan
The current situation lets your program (PocketSense in our case) access your transactions and positions directly without involving Yodlee, Quicken etc handling your data. Their announced intention is to require you to go via a third party that they work with to then provide the data to your computer or to a cloud app.
DeleteSee the official response in https://www.reddit.com/r/fidelityinvestments/comments/17tpgfr/why_doesnt_fidelity_support_ofx_downloads_anymore/?rdt=47495
So it would be more precise and robust than screen scraping, but would still require that your info go to a third party. It would be safer in that the third party would not have your Fidelity credentials. It also would provide an income stream to Fidelity. I don't begrudge them that, but I like to minimize the sharing of my financial data.
Thus the urgency to be able to generate OFX files, that can feed into PocketSense, from the history and positions CSV files that you can download from the Fidelity website. This is extra work over how PocketSense now works with Fidelity, but worth it IMO. I don't know if Vanguard will continue to offer OFX directly,
Cal Learner
A free Fidelity to OFX converter from Cal Learner (fidcsv3 or F2O name subject to change):
ReplyDeleteAmeridan's blog nicely documents where this program/project stands and explains the methodology.
See https://microsoftmoneyoffline.wordpress.com/2024/02/03/cals-fidelity-specific-csv-to-ofx-converter/
While the current main program is named fidcsv3.py, it will probably be renamed to something like F2O. py or f20.py at some point. Since this runs on windows, the filename will not be case-sensitive.
Advantages:
o open source
o developed on Python 2.7 as PocketSense uses
free
o can be used with PocketSense or without (I do recommend PocketSense)
o puts both transactions and positions and price info into the OFX files, just as with the OFX files that currently come in from Fidelity via PocketSense
o still being improved, so any bugs bug reports or suggestions posted here will be reviewed and probably acted upon
o the program finds your latest Fidelity positions and history CSV files in your Downloads folder without further interaction needed.
Disadvantages:
o not professionally coded.
o lots of global variables,
o not modular.
o the coding style evolved since being started March 2022, so method of doing the same thing is not constant thru the program(s)
o the code is not already openly published, but that will change if there is any interest.
human interface not polished, but will probably improve
o user must download the positions and history CSV files from the website for each run, but the program does find those in your Downloads folder without further interaction needed. Download from web is easy enough.
Questions and comments are welcome. I expect interest to pick up after the Fidelity OFX server disappears. But by that time, it would be too late for some things that you would miss out on that would would have made the operation initially easier.
Cal Learner
This is something that would be of interest to me. If I am willing to help if you need some.
DeleteJoe Quaranta
This comment has been removed by the author.
DeleteI have decided to open the Fidelity CSV to OFX (F2O) beta to all now. I
Deletehave a folder that should be accessible given the link. This is a folder
rather than a file:
https://drive.google.com/drive/folders/1-4fBbRxaPhNYZK0gNHuZ3n-Bdoncy2aR?usp=sharing
In the folder is f2o_beta3.zip which has several files. One file is
F2Obeta.doc
in Word 97 format, and F2Obeta.txt which is is the text-only version.
Feel free to post comments, bugs, or why did you do something the way
you did. The code evolved starting in March 2023.
This comment has been removed by the author.
ReplyDeleteThat worked. I have downloaded the zip file and extracted it but I didn't find any python files and the .doc file seems to be a binary file of some sort???
DeleteFirst, yes I did miss important files. The .doc file did work for me when I just tested. It is intended to be opened by Microsoft Word or other program that can read Word .doc files. I have added a *.txt version too.
DeleteNew method. This is a folder rather than a file:
https://drive.google.com/drive/folders/1-4fBbRxaPhNYZK0gNHuZ3n-Bdoncy2aR?usp=sharing
I will delete the post with the old link. The new method should mean that I don't have to create a new link to change content.
It contains f2o_beta2p.zip and a simple txt file that does nothing.
That worked. I'll try out the F2O_DBgather.py program along with the bat files. The doc file will open in ms word. On my system the .doc extension automatically opens in a text editor. Thanks
DeleteThe F2O_DBgather.py program worked on my data except for one ofx file (one generated from a cash management account which is just a checking account from Fidelity). It errored because the split_ofx list has only 1 entry in it. ( the ofx does not have any since it only contains a cash entry). Since the file F2O.py was not in the files shared with me, I can't proceed any further.
DeleteI will need to deal with that. Currently I have all of my cash in my cash management account used to purchase SPRXX. I should sell some of my SPRXX to generate an entry for the core position for testing.
DeleteI had intended for that F2O.py file to be there. It is there now with the same zip filename, but a little bigger.
Both programs (F20.py and F20_DBGather.py) worked well with my data other than the cash management account mentioned previously. Thanks, I think these will be useful in the future.
Deletedealt with it. f2o_beta3p.zip contains a fixed version of F2O_DBgather.py
ReplyDeleteIt also has a copy of F2O_DBgather.bat.bk1 which is my zip file, but is intended to be modified for your file structure. It is not called F2O_DBgather.bat in the zip file to avoid accidentally overwriting a batch file you already modified.
Besides the bug fix, F2O_DBgather.py accepts an optional second parameter (integer) that would set the DEBUG level. Default is zero.
I expect to open the beta pretty soon.
I have installed Anaconda, downloaded the scripts, and not having any luck getting started. When I double-click Setup.py, nothing happens. Next, I ran the Anaconda Prompt and typed in Setup.py. This gave me a couple of errors, one on import urllib2 and another on pyDes.py (print 'Invalid Password' - SyntaxError: missing parentheses). Any suggestions on how to get started? Thank you.
ReplyDeleteI am not familiar with Anaconda. I would start with getting the version. If you launch Python with the --version switch, it should give the version.
DeleteIf you launch python so that you get the Anaconda Prompt, I expect the sign-on message included the version.
If the version works out ( maybe 2.7.10 to 2.7.18), then I guess you may want to get a different distribution.
http://pocketsense.blogspot.com/2021/02/dev-testing.html does mention Anaconda, but it also includes a link for ActiveState.
Cal Learner
Has anybody else lost ofx server access to citi? My last successful Pocketsense call seems to have been Feb 9th....
ReplyDeleteCiti credit card worked fine this morning, but I don't have any bank accts with them.
DeleteSame here. Citicard OFX via PocketSense worked this morning. It has been nicely reliable for me.
DeleteThis is my sites.dat citicard descriptor:
<site>
SiteName : CitiCard
AcctType : CCSTMT #credit card
fiorg : Citigroup
fid : 24909
url : https://mobilesoa.citi.com/CitiOFXInterface
bankid :
brokerid :
ofxVer : 103
appid :
appver :
mininterval:
timeOffset :
</site>
In setup.py, doing the "2. List Accounts" choice, I do have ConnectKey: listed for each of two accounts. I did not do anything to set up that. When I do the rare move of my PocketSense to another folder, I copy my connect.key and ofx_config.cfg files over to the new folder.
While I have a Citi bank account, I don't think they provide an OFX server for that. I use their web interface to get a qfx file. I do that very rarely because it is a bit tedious. I make sure the dates I download do not overlap with the transactions in Money because they don't/didn't have the thing about having the FITID the same each time working.
Cal Learner
Thank you gents. My sites.dat setup matches yours Cal (and yes, i should have specified that this is indeed for a credit card which has worked beautifully up until now). It appears to be a credential issue, however. I do get a response from the ofx server with the following:
DeleteERROR
Information does not match with our record. Please make sure you select the correct Financial Institution and enter correct username/password.
I decided to nuke the connection and set it up again...and yeah, it chokes on my credentials (the same ones that work perfectly when logging in on the citi website). I am a bit stumped on this one if others are not seeing the same... oh well... I do have other ways of getting my citi transactions so it is not a big deal. But this means my pocketsense direct connections are down to just Fidelity and Amex... :S
Unless something has changed, I believe you need to enable direc access on the Citi site. Not at computer now, and don’t recall the details, but you add a new connection from an account menu and *then* add the account in PS during the ~ 10 min window. -Robert
DeleteThanks Robert; I suspected that may be the case, but I looked high and low on the citi website and could not find such a feature... No rush, but if you do find that menu option on your end please do report back. i did find some references to Quicken, but that is all in context of Web connect which of course is different.
DeleteOooo... Now that you say that, I kinda remember that discussion.
DeleteIn http://pocketsense.blogspot.com/2019/08/comments.html see "After making the sites.dat changes, I did not need to use the Setup.py script, but I did need to use the Citi 10-minute process to relink the PocketSense scripts to my Citi account. The easiest way to do that is to log in to https://citi.com/datamanagement. "
I cannot log into the website at the moment.
Cal Learner
Thank you Cal; that did the trick and I am reconnected through Pocketsense now. Maybe my authorization (prior one evidently from 2022) expired, so i wouldn't be surprised if others find that the connection stops working at some point. How one can find that critical page without knowing the direct link is still a mystery to me. Will make sure to save the link for posterity. Thanks again!
Deletedjevel, go to https://online.citi.com/US/ag/ofxConsentPage and authorize "Quicken Windows".
ReplyDelete-ameridan
Thanks Ameridan; I can confirm that your link goes to the same place as Cal's!
DeleteThe actual procedure has been on my settings page all along (for posterity) @ https://microsoftmoneyoffline.wordpress.com/2010/10/06/cal-learners-review-fidelity-401k-citi-card-and-vanguard-account-info/
ReplyDeleteAnyone use the PocketSense scripts to download Regions Bank account data? I've used them for years without issue. My downloads stopped working this past weekend and are still not working. It would appear to me that Regions as shut down their OFX server much like USAA did earlier this year. Just trying to verify that that's the case.
ReplyDeleteHi John,
DeleteHere in Georgia, using the PS scripts, Regions Bank is working as expected with both BASTMT and CCSTMT type accounts.
In sites.dat I set userAgent: none
Also delay: 3
I set the delay because I was getting bounced out midway through the downloads.
HTH -Kevin N.
I started getting an internal server error this past week. I don't receive an OFX response back when I try to download transactions or when I try to set up a new account. It's really strange.
DeleteHi John,
DeleteCheck to make sure your Profile at Regions has not changed.
Log on to Regions. Hover over 'Customer Service', select 'Settings'.
Click the 'Manage Profiles' link. Here, make sure the options for 'Online' and 'Quickbooks' are checked.
Click 'Save'.
Kevin - None of my profile info has changed. I tried doing a completely clean install on another machine to see if I could run the setup script to download just my Regions accounts. The script fails on that machine as well. I suppose Regions could have blacklisted my IP address as one of my OFX account requests for a new account that was setup in January was routinely failing. If they had done that though, I shouldn't be able to log into regions.com either.
DeleteKevin - Adding userAgent as "None" and adding a delay of 3 seconds seems to have resolved the issue.
DeleteI'm glad to hear that you were able to get this working again.
DeleteThank you for reporting back with your results.
I added kevin's recommended changes to my settings page. thx
ReplyDeletehttps://microsoftmoneyoffline.wordpress.com/2010/10/06/cal-learners-review-fidelity-401k-citi-card-and-vanguard-account-info/
Hi Dan,
DeleteThank you for maintaining your site. It is a trove of valuable information.
Just to clarify, my suggested 'delay' setting for Regions Bank and Regions Credit Card is 3 seconds. It was inadvertently entered as 30 seconds on your site.
Robert, any plans to publish a csv-2-ofx converter for PocketSense fans?
ReplyDeleteF2O, a Fidelity-specific CSV to OFX converter project is actively in progress. See https://microsoftmoneyoffline.wordpress.com/2024/02/03/cals-fidelity-specific-csv-to-ofx-converter/
DeleteThe user could select to have the generated Fid*.ofx files go to the PocketSense import folder, but that would best not be done until the Fidelity OFX server shuts down.
PocketSense would still consolidate the OFX files I think, but I don't think the scrubbers get run on the import folder files.
Cal Learner.
Regarding a csv converter integrated w/ the existing ofx scripts, I don't currently have such plans. I did write a simple conversion script for personal use last year, but it's limited to banks & credit cards and doesn't currently have a "configuration interface". More precisely, I wrote & use it only for Discover.
DeleteMore broadly, I see "csv to ofx" as a different animal than "fetching statements via ofx services". It's easier than ofx in many ways for banks/credit but seems tedious for investments, since there's no "standard" for how a firm may report transactions and/or positions. The F2O mentioned above sounds like Fidelity is providing a normalized interface to work with, but I'm not a Fidelity customer so haven't tested.
That's a long-winded answer vs a simple "no", but my thoughts at the moment :|
Is anyone having a problem with downloading transactions from Capital One credit card? I've never been able to use Pocketsense with this account but until today I have been able to manually download them and import the OFX file into Money. Now I'm getting a message from Money about an Invalid format. When I look at the file with Notepad I see this strange sequence of messages:-
ReplyDelete20240404000000.000
20240411000000.000
nullnullnullnullnull
I'm wondring if Capital One is the latest financial institution preventing customers getting their own transaction data?
Sorry , when I copied that data in all the OFX parameters disappeared. I'll try again:-
Delete"BANKTRANLIST"
"DTSTART 20240404000000.000 /DTSTART"
"DTEND 20240411000000.000 /DTEND"
"nullnullnullnullnull /BANKTRANLIST"
Today the problem fixed itself or maybe Capital One did something.
DeleteI am glad they fixed it. It was clear from that nullnullnullnullnull that the OFX file was not right.
DeleteFYI, if you put < symbols into posts, the blogspot software thinks those are the start of tags, and removes them up to the following > symbol.
If you replace all < symbols with these 4 characters < then it should post properly. I think it is not necessary to replace > symbols with > as long as this sentence makes sense (it says "replace > symbols).
Cal Learner.
@Matt,
ReplyDeleteI couldn't reply to your original post as comments there are no longer permitted.
I wanted to thank you for your work on porting Pocketsense the Python 3.
https://github.com/magnuspyke/ofxscripts
Due to Microsoft ending support for Win 10, I figured that I'd give Linux Mint a try.
After a few newbie failures, I was able to get Pocketsense working as expected in Mint.
Failures = not running Setup.py in Terminal, typing Python3 instead of python3 (cAsE sEnSiTiVe) and typing setup.py instead of Setup.py (cAsE sEnSiTiVe)
I know that I won't be able to run MS Money there, but I can and do run Moneydance there.
Thanks again.
-Kevin N.