Generally you owe taxes to the jurisdiction where you're performing the services and/or earning the money. This is called "sourcing" - income is sourced to where it's earned, and income earned for personal services is earned where the services are performed. Nonetheless, many (all?) jurisdictions also apply the concept of residency and tax their residents on their worldwide income, regardless of where it's sourced.
So, if you're a resident of New York, but you're performing personal services in California - you may end up being taxed by both: California, because the income is sourced to it, and New York, because you're its resident and it taxes your worldwide income.
California requires filing a non-resident return not based on the time spent in the State, but based on the total income. See the FTB 540NR booklet for the filing requirements. Generally, the rule of thumb is that if you're required to file a Federal tax return and you have income sourced to California - you need to file a California return as well.
Should I file and pay CA taxes for some portion of my salary and seek a refund from NY?
Some states provide credits for taxes paid in other jurisdictions, but not all. New York allows credits for taxes paid to other states, but not other countries. This is called "Resident Credit". Use the form IT-112-R (IT-112-C for credits for taxes paid to Canadian provinces) to claim.
Should Google be witholding CA taxes when I badge in at the Mountain View campus? ? Is there any employer in the country that even does this correctly, or is this something where technically huge numbers of people are in noncompliance but there's no enforcement?
Not your problem, employer payroll taxes is a very complicated business and there are companies whose whole business model is to handle this for employers.
Like if I take my family on vacation in California and do a little work while out there, would this also then affect my state taxes?
Potentially, yes. This is true for any cross-boundary work, California is neither special nor unique.