(comments closed)
Last Updated: 8/27/2017
Recent changes at Vanguard prompted another round of minor patches to the scripts, which can be downloaded @ https://sites.google.com/site/pocketsense/home/msmoneyfixp1/p2.
Last Updated: 8/27/2017
Recent changes at Vanguard prompted another round of minor patches to the scripts, which can be downloaded @ https://sites.google.com/site/pocketsense/home/msmoneyfixp1/p2.
Detail: Vanguard is now using session tokens (cookies). There were a number of ways to handle it, but I chose to allow the first connection attempt to fail, and return the server-response cookie on the next attempt, thus using the same tokens as the server. We could instead send a client token on the first pass, and skip the double-connect, but that requires an assumption re: expected server tokens. The hope is that the chosen method will support changes at other banks, should they choose to use session keys, regardless of token schema.
Thank you again, update works!
ReplyDeleteKale
Setting a timeOffset in sites.dat, such as
ReplyDeletetimeOffset : -30
gives an error "type object 'datetime.datetime' has no attribute 'timedelta'"
A cure is to change "from datetime import datetime" to "from datetime import datetime, timedelta" and to change "datetime.timedelta(hours=h)" to "timedelta(hours=h)".
I don't know why that is needed.
CL
What version of Python are you using?
DeleteNever mind... it's a bug introduced somewhere along the way. I replaced datetime.datetime calls w/ a datetime import. Missed the timedelta part, since I don't use it. Will fix.
DeleteThanks for all the work you have done.
ReplyDeleteThere is an issue i am facing. I am using Google finance and not yahoo finance. If i check for quotes for symbol "BOM:500012", the result given is wrong i.e. i think it is multiplying the code itself i.e. 500012 * the day's price and giving wrong result. e.g. today's rate is 32.05 but getdata is returning like G: BOM:500012 16025384.6 8/24/2017 4:00pm
I see the problem, and you're correct re: the multiplier. The scripts allow a multiplier option for each quote line, and is specified using a M:# parameter. The BOM: is being interpreted incorrectly, so I'll modify the script to handle it better.
DeleteThanks. The update works fine now.
DeleteThe scripts have been updated to address the two issues mentioned above.
ReplyDeleteI finally got my acct defined properly with USAA, at least it said it downloaded successfully. But I get this error message:OFXAdapter: Failed to parse request: Unable to parse a composite field "OFXRequest.OFXRequest": postTag "/OFX" not found. I can see no OFX data when I open the XFER file other than that msg. Any idea where I go from here?
ReplyDeleteJust a guess here, but try explicitly setting ofxVer:102 in sites.dat for USAA. If that doesn't work, see if they support ofx 2.x (e.g., ofxVer:211).
DeleteNo, that did not work. Msg: OFXAdapter: Failed to parse request: Unable to parse a composite field "OFXRequest.OFXRequest": postTag "/OFX" not found.
ReplyDeleteI will ask them if they support 2.X
I did a quick search and found another comment from long ago: "Apparently, with USAA, leading 0's in an account number are really necessary". Just a fyi, since many banks/sites require leading zeros be included w/ the account number (e.g., 00001987567).
DeleteYes, I already knew that from my past experience with Quicken. I use two leading zero's. I also found that if I did not get the acct number correct, the error msg tells me that. But in this case, it says the data downloaded, even though I get the error msg above.
ReplyDeleteI have multiple accounts at USAA that are working, here's the USAA section from my sites.dat:
ReplyDeleteSiteName : USAA
AcctType : BASTMT #bank
fiorg : USAA
fid : 24591
url : https://service2.usaa.com/ofx/OFXServlet
bankid : 314074269
brokerid :
appid : QWIN
appver : 1700
mininterval:
timeOffset :
I also noted that usernames must be zero-padded to exactly 9 digits and account numbers must be zero-padded to exactly 10 digits.
Hope that helps!
Thank you. That is for USAA Federal Savings Bank, I am trying to get it to work for USAA Investment Management Co. which uses different stuff. I did know that the user ID needed to be 9 digits with leading zeros. I wonder if I can dummy in the appid and appver?
ReplyDelete
ReplyDeleteLarryB, what is your sites.dat entry for USAA Investment Management Co?
CL
Name: USAA Investment Mgmt Co
DeleteAcctType: INVSTMT
fi: USAA
Fid: 24592
url: https://service2.usaa.com/ofx/OFXServlet
brokerid USAA.com
Is that a copy/paste from the actual sites.dat file? Some field names aren't what's expected. Specifically: Name should be SiteName and fi should be FiOrg. Also, the colon is missing after brokerid.
DeleteIt is not a cut/paste since it is on a different computer. It is all entered properly in the form on the sites.dat file
ReplyDeleteHi Robert! Got a question. For accounts that use the same bank, how hard would it be to set it up to get all files on the same login session? I have just changed the password on BoA where I have multiple accounts. I forgot to change the password on PocketSense, so it issued an incorrect login multiple times... boom. BoA locked me out. My fault entirely, but it would be less likely to happen if only one login was used for all accounts in the bank.
ReplyDeleteI'm pretty sure the OFX standard supports it, but ... the scripts are written w/ an explicit "one connection per statement" logic. I like the idea, but I'd need to think it through. I'm certain that it can't be done "as is", and that some core logic would need to be revamped. The nice thing about "one connection per request" is that each account is processed independently, and therefore no assumptions re "grouping by sign-on credentials".
DeleteHas anyone been using Pocketsense with what used to be Hewitt Associates and is now Alight Solutions? I have been getting an error message "EOF occurred in violation of protocol" for about a month now, and I assume that they have changed things at their end. However, they have been remarkably and breathtakingly unresponsive in telling me what changed, so I wondered if anybody here had been using them and knew what to change to make it work again?
ReplyDeleteHarold
In Windows Internet Options, in the Advanced tab, make sure that Use TLS 1.0, Use TLS 1.1, and Use TLS 1.2 are all enabled.
ReplyDeleteCL
I already had it set that way, but I tried turning it off and on... didn't fix it. Which means the problem still persists...
DeleteI upgraded to Active Python 2.7.13, and that solved the problem.
DeleteMy BofA accounts stopped working this week (my last successful run was last week) with the error "An incorrect username/password combination has been entered. Please try again." Neither the username nor the password has changed and I was able to successfully login via their website as usual. I have a credit card and 3 checking accounts I use with PocketSense. The downloads have been working perfectly for a couple of years. Any ideas before I try to call BofA and see if my OFX access was disabled?
ReplyDeleteSame issue with BoA here. Last successful download was Sept 23.
ReplyDeleteJust a FYI for folks having issues... be sure you've updated to the latest Python 2.7.x version.
ReplyDeleteDo you mean the updated scripts (from 8/27) or upgrading Python itself? I did download the 8/27 scripts yesterday and am still getting the same error.
DeleteTO ALL BOA CUSTOMERS:
DeleteChange the ODXVER in Sites.dat to 103 and give it a go.
The Bank is clueless and helpless as usual.
Ray
I added this line to the BofA site profile in sites.dat but the error remained the same:
DeleteofxVer : 103
I was referring to Python itself, since there were one or two versions that caused SSL connection issues. Probably not related in this case, but a good idea.
DeleteI upgraded to Active Python 2.7.13, and that solved my problem with Hewitt Associates / Alight Soutions that began on 8/17/17 (presumably due to some kind of library upgrade on their side). Thanks!
DeleteThank you Anonymous.
ReplyDeleteofxVer: 103 worked for me in solving BoA issues.
It worked on my BoA personal accounts, BoA business accounts (different bankid) and BoA credit card. I'm using ActivePython 2.7.1.4
Robert - are the order and spacing of the sites.dat entries critical? Don't know if that might have affected B-Man and why 103 didn't work for him.
Here is my working entry:
SiteName : BOA BANKING
AcctType : BASTMT
fiorg : HAN
fid : 5959
url : https://eftx.bankofamerica.com/eftxweb/access.ofx
bankid : 052001633
brokerid :
ofxVer : 103
appid :
appver :
mininterval: 30
Field order doesn't matter, but probably a good idea to cross-check all parameters vs your working set.
DeleteI updated to ActivePython 2.7.13.27 and still same error. Here's my Sites.dat entry for BofA:
DeleteSiteName : BOFACHECK
AcctType : BASTMT #bank
fiorg : HAN
fid : 5959
url : https://eftx.bankofamerica.com/eftxweb/access.ofx
bankid : 053000196
brokerid :
ofxVer : 103
appid : QWIN
appver : 2300
mininterval:
timeOffset :
moneydance's config file is located at http://moneydance.com/synch/moneydance/fi2004.dict
DeleteThis is their entry for BofA:
{
"id" = "md:1039"
"access_type" = "OFX"
"fi_name" = "Bank of America"
"bootstrap_url" = "https://eftx.bankofamerica.com/eftxweb/access.ofx"
"fi_id" = "5959"
"uses_fi_tag" = "y"
"fi_org" = "HAN"
"app_id" = "QWIN"
"app_ver" = "2400"
}
It looks like you need to set appver to 2400. The bankid looks like it is the ABA routing number for your account, double check that yours is correct. I also do not use BofA.
What I had to do to get BofA downloading again:
Delete1. ofxVer : 103
2. run Setup.py, test accounts, test BofA, it fails.
3. Use my browser to login to bankofamerica.com, it will prompt you with a below message.
4. Click OK, it will start the 10 minutes countdown.
5. Go back to the command prompt where I have my Setup.py still running, test my BofA account again (within the 10 minutes). It works.
--------------------------
Thanks for signing in.
We want to make sure you stay protected when you link your Bank of America account to financial management software.
To stay secure, you’ll need to sign in to Online Banking when you:
◾Use Quicken or QuickBooks on an unrecognized device
◾Add a new Bank of America account to record transactions
◾Receive a software update for Quicken or QuickBooks
Now that you’ve signed in, you can go back to Quicken or QuickBooks and continue to download your transactions.
Please note: You’ll need to sign in to Quicken or QuickBooks within 10 minutes of selecting OK or you’ll have to repeat this sign-in process at another time.
Whew! I got the prompt when I signed in through BofA OLB and now the downloads are working. Thanks all!
DeleteLooking at the GnuCash site OFX_Direct_Connect_Bank_Settings page, you might try removing your appid or using "appid: QBW".
ReplyDeleteI don't use BofA. Just an idea.
CL
Thanks for the continued suggestions all. Nothing's helping, though. Same error when I removed the appid and when I changed it to QBW.
DeleteThanks to all for the help with the recent issue on Bank of America! Keep up the good work.
ReplyDeleteI got my BOA checking/savings accounts working by adding in the ofxVer : 103 line and then logging into the website and approving the access. However, I still can't get my BOA credit card to work. It just gives the old username/password error. Here is my entry.
ReplyDeleteSiteName : BOACC
AcctType : CCSTMT #credit card
fiorg : HAN
fid : 5959
url : https://eftx.bankofamerica.com/eftxweb/access.ofx
bankid :
brokerid :
ofxVer : 103
appid :
appver :
mininterval:
timeOffset :
I've never been able to get BofA checking and savings accounts to download through PocketSense even though I've had no trouble with their credit cards. Can you tell me what OFX settings you're using that work for you?
DeleteKen, I believe I added his BOA settings in my blog page.
Deletehttps://microsoftmoneyoffline.wordpress.com/2010/10/06/cal-learners-review-fidelity-401k-citi-card-and-vanguard-account-info/
Thank you! Your settings worked after I changed the BankID to the ABA number for my accounts. You may want to make a note that one needs to check the ABA number, which changes for different States. Otherwise the settings worked perfectly.
DeleteDone.
DeleteThe CC is working now. I had not added the ofxVer:103 line until after I had done it on the checking/savings and tested those. When I ran it again they all errored out with the 'or log on' error so I logged on to the website, gave permission, and now everything works.
ReplyDeleteAnother BoA update. ofxVer initially worked for me, and then today I started getting invalid username/password errors. Login on via the web site gave me the Quicken permissions prompt, and then everything went back to normal.
ReplyDeleteBottom line - there is something BoA is doing which may cause you to loose Quicken access. So first order of defense, if PocketSense fails, immediately log unto web interface to make sure you give permissions.
I'm not sure if they are now sensitive to IP changes or some other variable. This might make sense, since they are effectively trying to set up a two-step initial authentication.
I started getting this error for my Citi Card account:
ReplyDelete** An ERROR occurred sending POST request to www.accountonline.com
Exception type :
Exception val : [Errno 1] _ssl.c:507: error:1407742E:SSL routines:SSL23_GET_SERVER_HELLO:tlsv1 alert protocol version
An online error occurred while testing the new account.
Anyone else?
Citicard works for me. It worked as scheduled overnight, and I just tried again as a test using setup.py.
DeleteMy site descriptor is simple:
==========
SiteName : CitiCard
AcctType : CCSTMT #credit card
fiorg : Citigroup
fid : 24909
url : https://www.accountonline.com/cards/svc/CitiOfxManager.do
bankid :
brokerid :
appid :
appver :
mininterval:
timeOffset :
================
That has been my setting too, but since Sunday it hasn't worked. All of my other accounts are working though.
DeleteI get the same error on my Citicard and Searscard, also since Sunday...
DeleteI have a Universal AT&T card, which is also Citibank, and uses the same server. Just tested, and works fine. Site settings are the same as above.
DeleteThe error listed above is consistent w/ what's been seen when running an older version of Python, but the server updates to a newer version, which triggers an SSL exception. So, the first thing is to verify that you're running the latest Python release [2.7.13].
Thank you so much for the reply Robert! I was at 2.7.6 and I’ve upgraded to 2.7.14, which is apparently the latest as of Sept 16th. It resolved the issue!
ReplyDeleteDan (ameridan)
Discover Bank has found yet another new way to screw up their FITIDs. The latest new format is:
ReplyDeleteYYYYMMDDSDFCCCCCCXXXXXXXX
YYYY year posted
MM month posted
DD day posted
SDF literal
CCCCCC six digit non-unique code. two checks with different amounts posted on same day have same code
XXXXXXXX eight digit sequence that changes daily (transaction has different sequence on next day)
Below is a context diff of the change required to get Discover Bank working again. This change is
in _scrubDiscover_r1() in scrubber.py. NOTE: blogspot strips the required indentation
#strip the serial value for credit card transactions
if accType=='CCSTMT':
bx = len(fitid) - 5
fitid_b = fitid[:bx]
+ #strip the serial value for bank transactions
+ else:
+ bx = len(fitid) - 8
+ fitid_b = fitid[:bx]
#find a unique serial#, from 0 to 9999
seq = 0 #default
while seq < 9999:
fitid = fitid_b + str(seq)
Thank you Andrew. I made the changes, and then realized I only use Discover Card, not Discover Bank, so I am not seeing the problem. Nevertheless, the scrubber still works.
DeleteFollowing are experimental changes I have made to populate the check number field for check withdrawals.
ReplyDeleteIn _scrubDiscover() in scrubber.py. NOTE: blogspot strips the required indentation and "{" and "}" need to be replaced by "<" and ">"
_sD_knownvals = [] #reset our global set of known vals (just in case)
+ # insert a line break before each transaction for readability
+ # this should also ensure that the regexes below don't match across transactions
+ p = re.compile(r'({STMTTRN})',re.IGNORECASE)
+ ofx_final = p.sub(lambda r: _scrubDiscover_r0(r, accType), ofx)
+ ofx = ofx_final
+
#regex p captures everything from {FITID} up to the next , but excludes the next "{".
#p produces 2 results: r.group(1) = field, r.group(2)=value
p = re.compile(r'({FITID})([^<\s]+)',re.IGNORECASE)
#call substitution (inline lamda, takes regex result = r as tuple)
ofx_final = p.sub(lambda r: _scrubDiscover_r1(r, accType), ofx)
+ #regex p captures everything from {TRNTYPE}DEBIT up to the next "<" after the {NAME}Check tag and field.
+ # Discover Bank codes checks as
+ # {STMTTRN}{TRNTYPE}DEBIT{...}{NAME}Check############{/STMTTRN}
+ #p produces 4 results:
+ # r.group(1) = {TRNTYPE}DEBIT,
+ # r.group(2)=stuff up to next "{NAME}Check "
+ # the \s keeps it from matching across transactions due to the added \n since regex will try to make the longest match
+ # r.group(3) = "{NAME}Check ",
+ # r.group(4) is the check number
+ # The result should produce a entry that will import the check number in Money
+ # {STMTTRN}{TRNTYPE}CHECK{...}{CHECKNUM}############{NAME}Check{/STMTTRN}
+ if accType=='BASTMT':
+ ofx = ofx_final
+ p = re.compile(r'({TRNTYPE}DEBIT)([^\s]+)({NAME}Check[ ]+)([0-9]+)',re.IGNORECASE)
+ #call substitution (inline lamda, takes regex result = r as tuple)
+ ofx_final = p.sub(lambda r: _scrubDiscover_r2(r, accType), ofx)
+
return ofx_final
Two new added functions
+ def _scrubDiscover_r0(r, accType):
+ fieldtag = r.group(1)
+ return '\n' + fieldtag
+
+ def _scrubDiscover_r2(r, accType):
+ #regex subsitution function for _scrubDiscover()
+
+ trntype = r.group(1)
+ rest = r.group(2)
+ name = r.group(3).strip(' ')
+ checknum = r.group(4)
+ return '{TRNTYPE}CHECK' + rest + '{CHECKNUM}' + checknum + name
+
Thanks Andrew. If you have a chance, can you send your new scrubber.py module to pocketsense1 /at/ gmail /dot/ com? I won't be able to look at it for a couple days, but that will save reformatting. I don't use a Discover Checking account, so I can't test myself.
ReplyDeleteIt looks like Yahoo is futzing with there CSV server. It's throwing an error 'An error occurred connecting to the Yahoo CSV service'.
ReplyDelete-Kevin N.
I noticed that recently, but no issues w/ the Google service.
DeleteHow do I get the Google service to work? I have "EnableGoogleFinance: Yes" but it does not solve the problem. Thank you.
ReplyDeleteI found my prices were up to date. I had thought those were coming from Yahoo. Maybe not.
ReplyDeleteIn my sites.dat, I have these:
----------
#YahooURL: http://ca.finance.yahoo.com
QuoteCurrency: USD
EnableYahooScrape: Yes
YahooTimeZone: -5:EST
ShowQuoteHTM: No
AskQuoteHTM: Yes
---------------------------
My *.py files are dated august 2017. That indicates that I have the latest released Pocketsense. I don't know why my prices that are not from OFX files are working, but of course I am glad they are.
CL
Like Kelvin I get "An error occurred connecting to the Yahoo CSV service". My sites.dat is similar to CL's except #QuoteCurrency: USD.
ReplyDeleteI also tried quotes.py from Harry Sit (The Finance Buff) but no luck
GG
I am seeing the same thing. Last good update of stock prices was yesterday, 11-1
ReplyDeleteDD
I plugged the url into my browser and got the following message: It has come to our attention that this service is being used in violation of the Yahoo Terms of Service. As such, the service is being discontinued. For all future markets and equities data research, please refer to finance.yahoo.com.
ReplyDeleteUPDATE - just turn off the Yahoo site in the sites.dat file. It tries Yahoo first if both Google & Yahoo are enabled. By turning off Yahoo it goes to Google and works fine.
DeleteUnfortunately, the Google finance page often has very wrong prices or the lookup for a particular ticker (especially leveraged ETFs and not so well known ETFs) goes to the wrong equity requiring corrections. Perhaps the Yahoo code can be replaced with Yahoo web page scraping rather than the web API. There are no restrictions on scraping their finance page itself.
DeleteI see that the sites.dat file has
DeleteEnableYahooScrape: Yes # Try a screen scrape if the primary Yahoo csv
Shouldn't this work if CSV fails?
Regarding quote lookup on Yahoo and Google:
ReplyDelete1. There is currently a Yahoo screen scrape routine in the scripts, should the csv interface fail. However, the scrape routine doesn't work w/ the current Yahoo site. Look in the sites.template for the latest available settings, but the following two are important here:
EnableYahooFinance: No # Enable quote lookup on Yahoo Finance
EnableGoogleFinance: Yes # Enable quote lookup on Google Finance
2. I've personally had no issues w/ Google Finance quotes, and their database has proven much more extensive. However, the symbols used by Google don't always match what is used by Yahoo. You must lookup the actual ticker you want, and then get the exact symbol from the header. For example, the S&P500 ticker on Google is "INDEXSP:.INX", or more simply ".INX". To be precise, you can use the fully qualified Google ticker, but for common US tickers you don't need the prefix.
3. If you need to map a Google symbol to an existing symbol in Money, you can use the s: option when defining the ticker. Using the example in #2, you would add "INDEXSP:.INX s:^GSPC", so that the quote is grabbed for the fully qualified ticker, but "^GSPC" is passed to Money as the symbol.
I'll try to look again at the Yahoo scrape option, and see where it stands. I'm about to put out a patch for the recent Discover Bank numbering issue, and may be able to include this as well. The thing to remember about scraping is that it's UNRELIABLE unless the site uses consistent/reliable encoding. Yahoo previously did not, but on first glance, they may have changed their ways (?).
Ok thx. Many symbols need to be fully qualified with the exchange prefix for using Google Finance. For example, NYSEARCA:FSTA. Just using FSTA for the ETF will download for Fuller, Smith and Turner PLC quote. Similarly unqualified SHYG will download for SHYG on the London Stock Exchange, not the ETF on NYSE. Mutual funds seem to work OK for the most part as will most 5 letter symbols because there is no ambiguity across exchanges.
DeleteUnderstand the issue with scraping. Hope there is a solution to avoid having to sanitize every symbol as above but if that is what is needed ....
Robert, Thanks for this great answer. I had never tried the scraping option. Do the currencies work with GoogleFinance? I set EnableYahooFinance to No, and currencies failed.
ReplyDeleteTake a look at finance.google.com . On the right, you should see some example currency quotes. Click the link, and you'll see the ticker in the header. Example: EUR/USD ticker is EURUSD. The same pattern is used throughout their currency tables (i.e. USD/JPY == USDJPY). Should be mostly the same as Yahoo, except Yahoo had a "=X" suffix, iirc.
DeleteThis comment has been removed by the author.
ReplyDeleteThanks Robert, for the currency tip. I didn't realize =X was part of the Yahoo quote and not a special code of some sort.
ReplyDeleteI'd also forgotten that currencies get downloaded as a "quote" and don't update Money's currency info. Oh well.
In any case I can't believe that seven years later we are still able to use Money thanks to your scripts. On the other hand, not everything else is as lasting. You may want to remove YHOO from the sites.dat template :)
-- Carlos
Hi Robert,
ReplyDeleteI noticed the following at Google.finance regarding personal portfolios...
"Google Finance is under renovation. As a part of this process, the Portfolios feature won't be available after mid-November 2017. To keep a copy, download your portfolio."
Do you think that this may have any effect on the scripts' ability to retrieve stock quotes from Google?
-Kevin N.
Should be fine, but "we shall see".
DeleteI have been using Pocket Sense for years. It stopped working last week. I was only building a QFX file to import into my 'Qoutes' account. I am willing to pay for some help (via paypal). I don't understand you to use this forum or start a new thread.
ReplyDeleteSee post above on 03-Nov-2017 describing how to enable Google quotes and *disable* Yahoo. The Google Finance option works well for situations I've tried. It was implemented specifically because it provided quotes for a broader range of global exchanges, and as a backup for the Yahoo service.
DeleteIf you want to wait a bit longer, I'm looking at updating the Yahoo quote interface to work w/ their new finance site. I haven't had time to focus on it, so can't say for sure how it'll go or when, but it seems promising.
Still lost. I have been using an OLD version of Pocket Sense. It failed since Yahoo changed so I am trying to start over. I am trying to
Deletefollow your instructions "OPTIONAL: Create an Account in Money to Receive Stock Quotes". In Money my dummy account is 'Quotes'.I turned off EnableYahooFinance (and YahooScrape) and I can collect prices BUT only if I do it from Setup.py but it will not build a good QFX file. If I try to start GetData.py I get "No accounts have been configured. RUn SETUP.py to add accounts. Do I need to create a 'Quotes' account to match my Money file? I have tried to follow the instructions but I just don't get it.
If you think you can help me then let's use TeamViewer to get the job done. I will gladly buy you a stake dinner for two.
#1: Enable Quotes in Setup (option 6).
DeleteSome of the mobile apps that also got perturbed by Yahoo change have switched to https://iextrading.com/developer/docs/#iex-api-1-0
ReplyDeletefor quotes. Perhaps, that is more robust/stable than scraping Yahoo web site.
Pretty sure their api only works w/ stocks traded on the IEX exchange, which is a small universe. Yahoo and Google are hooked into all the exchanges, mutual funds, commodities, etc.
DeleteThere is a problem with quotes from Google for mutual funds. The date of the quote is wrong and so it doesn't update Money prices properly. For example, all mutual funds right now are reporting as Nov 5. You can verify this by going to the Google Finance page as well and noticing the date below the quote. How is this working for others? Or am I doing something wrong?
ReplyDeleteHi,
DeleteI'm seeing the same thing. At first I thought that these were 2-day-old quotes but as you've discovered, the quotes appear to be current but the date is off.
-Kevin N.
I see the same, but it only seems to apply to class=MUTF quotes (US mutual funds)? The price is the latest, but the quote time is a day behind. Money should still reflect the "latest" value, though, since it will use the "last quote" for pricing, regardless of the date.
DeleteIt would be easy to correct in the scripts, *but*, it's a bug in the Google code that's likely to get fixed. If so, the script would then be wrong :(
Hi Robert,
DeleteYou're correct, the date discrepancy appears to only apply to Mutual Funds.
I've only recently started using Google for quotes. Like you say, the values should still reflect current values regardless of the date.
By a 'fix' do you mean something similar to the current m:value or s:value options? If so, that switch could be shut off if and / or when Google squashes their bug.
Then again, if the values are correct then there isn't much need for a fix. Carry on. :)
-Kevin N.
Unfortunately, Money is NOT updating to the "latest value" for me requiring a manual update for every MUTF. I wonder if the fact that there is a manual update already prevents the update a day or two later with that same date. For example, I manually corrected the fund prices for Nov 11 yesterday in Money. Now, if I update at the end of the day today and Google, say for example, reports the date as Nov 11 but with Nov 12 prices, will Money update the value for Nov 11 that already exists with the Nov 12 price? I don't think this is happening. There may be something else wrong (say what time the script is run) but I am definitely not seeing (from looking at the reported price as well as the price history in manual price update dialog) Money update the prices to current value but with a previous date. So, it requires a manual update every time.
DeleteMoney will not overwrite an existing quote entry for a given date during import. If a person runs the script ten times in a day, for the same date, it only imports the first one. If you enter a quote price manually for a given day, it will not be updated by an import.
DeleteSo if I'm trying to get good settings in Sites.dat, what do I need to do between tests?
ReplyDeleteIn other words - try settings a, b, and c. Failure, going to change c to d.
Assuming in that case that the failed new account needs to be deleted and recreated, but do I also need to exit and relaunch setup.py?
Account info configured within Setup will update as you go. Data changed in sites.dat is loaded when setup first starts.
DeleteThanks Robert. Do you mean that if I change a fiorg in the sites.dat, exit and relaunch setup, and test an existing account that used that entry it would update and use the new fiorg?
Delete>So if I'm trying to get good settings in Sites.dat, what do I need to do between tests?
ReplyDelete>In other words - try settings a, b, and c. Failure, going to change c to d.
>Assuming in that case that the failed new account needs to be deleted and recreated, but do I also need to exit and relaunch setup.py?
To simplify things, I would use the test feature of setup.py. Select to Test an account. Then choose the number for "Stock/Fund Prices" as the account.
Test Stock/Fund Pricing Updates (Y/N)? y
Open in the default browser (y/n)?y
Send the results to Money (y/n)? n
With that "n" for sending the results to Money, your money file will not be affected. Test as many times as you want. Your Money file will not be affected if you choose to not send the results to money. Money generates a new quotes*.ofx and a quotes.htm each time you do that. At least I think that is how it works.
CL
Other way around - when is sites.dat read? Do I have to exit and relaunch for my changes to be reflected?
DeleteOr am I looking at it wrong and sites.dat is always reread and I don't even have to delete the account, just make my changes to the file in notepad and just try the test again to see the changes if my acct number/user/pass didn't change?
As long as you save the file after editing, yes, it is reread.
DeleteFor quotes, it seems that Yahoo's json api is still working
ReplyDeletehttps://www.bogleheads.org/forum/viewtopic.php?f=10&t=218779&newpost=3609317&view=unread#p3609317
>Other way around - when is sites.dat read? Do I have to exit and relaunch for my changes to be reflected?
ReplyDeleteTo test this, I added an additional stock to the list in sites.dat, without closing setup.py.
I again select to test the quotes. The values got updated, and the new stock was added to the list of fetched quotes.
CL
>Or am I looking at it wrong and sites.dat is always reread and I don't even have to delete the account, just make my changes to the file in notepad and just try the test again to see the changes if my acct number/user/pass didn't change?
Fetching quotes does not involve what I would call a real account. For broker or credit card accounts, you are fetching OFX files. Yes OFX files can include prices for securities held. My experience is that these are only update by the broker with closing prices and those updates are posted during the night.
For testing out account access, rather than quotes, I would look at the OFX files in XFR without sending them on to Money. A bad password will result in a small OFX file. Other errors would result in small files I would think also. Expect that file to have the word ERROR in it.
Hi Robert,
ReplyDeleteIn your quest of fixing the Yahoo interface, I would like to bring to your attention that Yahoo has a json api which seems to be working:
url = "http://json-csv.com/?u=http://query1.finance.yahoo.com/v7/finance/quote?symbols=%s" % self.ticker
Someone at the bogleheads forum posted his/her Python code to retrieve quotes through this interface:
https://www.bogleheads.org/forum/viewtopic.php?f=10&t=218779&newpost=3609317&view=unread#p3609317
You might want to take a look.
I posted this information yesterday: AnonymousNovember 8, 2017 at 2:23 PM but it was mixed up with another thread.
Thanks, and yes, I have the json interface on the short list, and have a parser ~ ready.
DeleteI'm skeptical of anything that has a version# in the path, however ;) So... I was looking at adding a parser for the regular site as a backup. They now encode quotes into the web page (json also), but it's scattered, and filled w/ similar info for other quotes being displayed.
I've been short on time, so may just put out the json data api, and add the site backup later. They may use the json interface for their own feeds, so if they keep the basic format the same, the sites.dat YahooURL option would allow folks to update the script setting if/when the Yahoo path changes.
I added a new post describing recent changes and issues, so I'm going to close out comments on this one.
ReplyDelete