Python 3 Firestore DocumentReference.update fails with integer key in update data

[jdubinsky]

OS

OSX 10.13.4

Python version

Python 3.6.5

Library version

$ pip show google-cloud-firestore
Name: google-cloud-firestore
Version: 0.29.0
Summary: Google Cloud Firestore API client library
Home-page: https://github.com/GoogleCloudPlatform/google-cloud-python
Author: Google LLC
Author-email: [email protected]
License: Apache 2.0
Location: /Users/jdubinsky/.virtualenvs/firestore_th/lib/python2.7/site-packages/google_cloud_firestore-0.29.0-py2.7.egg
Requires: google-api-core, google-cloud-core
Required-by: firestore-th

Stack trace

>>> status_doc.update({'14': {'status': 'active'}})
Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/google/gax/api_callable.py", line 376, in inner
    return a_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/google/gax/retry.py", line 68, in inner
    return a_func(*updated_args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/grpc/_channel.py", line 484, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/usr/local/lib/python3.6/site-packages/grpc/_channel.py", line 434, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
grpc._channel._Rendezvous: <_Rendezvous of RPC that terminated with (StatusCode.INVALID_ARGUMENT, Invalid property path "14".)>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/site-packages/google/cloud/firestore_v1beta1/document.py", line 371, in update
    write_results = batch.commit()
  File "/usr/local/lib/python3.6/site-packages/google/cloud/firestore_v1beta1/batch.py", line 135, in commit
    transaction=None, options=self._client._call_options)
  File "/usr/local/lib/python3.6/site-packages/google/cloud/firestore_v1beta1/gapic/firestore_client.py", line 851, in commit
    return self._commit(request, options)
  File "/usr/local/lib/python3.6/site-packages/google/gax/api_callable.py", line 452, in inner
    return api_caller(api_call, this_settings, request)
  File "/usr/local/lib/python3.6/site-packages/google/gax/api_callable.py", line 438, in base_caller
    return api_call(*args)
  File "/usr/local/lib/python3.6/site-packages/google/gax/api_callable.py", line 380, in inner
    gax.errors.create_error('RPC failed', cause=exception))
  File "/usr/local/lib/python3.6/site-packages/future/utils/__init__.py", line 419, in raise_with_traceback
    raise exc.with_traceback(traceback)
  File "/usr/local/lib/python3.6/site-packages/google/gax/api_callable.py", line 376, in inner
    return a_func(*args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/google/gax/retry.py", line 68, in inner
    return a_func(*updated_args, **kwargs)
  File "/usr/local/lib/python3.6/site-packages/grpc/_channel.py", line 484, in __call__
    return _end_unary_response_blocking(state, call, False, deadline)
  File "/usr/local/lib/python3.6/site-packages/grpc/_channel.py", line 434, in _end_unary_response_blocking
    raise _Rendezvous(state, None, None, deadline)
google.gax.errors.InvalidArgumentError: InvalidArgumentError(RPC failed, caused by <_Rendezvous of RPC that terminated with (StatusCode.INVALID_ARGUMENT, Invalid property path "14".)>)

Repro steps

• Given a python dictionary with an integer key, call document.update
• For example, the update data could be {'14': 'active'}
• Call document.update({'14': 'active'})
• Should result in the above stack trace.
  (I reported a similar python 2 issue that got fixed in the last release Firestore DocumentReference.update fails with integer path #4320)

Code sample

>>> client = firestore.Client(project=project, credentials=creds)
>>> status_doc = client.collection('status').document('us-9')
>>> status_doc.update({'14': {'status': 'active'}})

Results in the stack trace above.

However, the following works:

>>> status_doc.update({'fourteen': {'status': 'active'}})
update_time {
  seconds: 1528994135
  nanos: 178078000
}

And a set call with the same integer string works:

>>> status_doc.set({'14': {'status': 'active'}})
update_time {
  seconds: 1528993682
  nanos: 216089000
}

[tseaver]

google.cloud.firestore_v1beta1._helpers.pbs_for_set only canonicalizes field names if the merge flag is passed as True

AFAICT, the canonicalization should happen whether or not merge is occurring.

The canonicalization is needed only to set up the field mask when merging.

[tseaver]

document.update(collection_path, document_id, {'14': 'active'}

Hmm, that isn't a correct invocation of document.update: it takes only the update data (the dictionary), plus an optional options argument.

The example you posted before that is correct. However, I'm unable to reproduce: see PR #5895, which adds a system test calling document.update({'14': {'status: 'active'}}).

Can you reproduce with the current release of google-cloud-firestore (0.29.0)?

[tseaver]

The new system tests I added in PR #5895 are passing.

[tseaver]

@jdubinsky Please feel free to reopen if you can suggest a reproduction case different than the new system test added in 9dc2514.

[jdubinsky]

@tseaver Those tests look valid to me. Here's some more info just to clarify (I've also updated the invalid repo step (it was from some custom code written on top of the firestore library):

>>> client = firestore.Client(project=project, credentials=c)
>>> client
<google.cloud.firestore_v1beta1.client.Client object at 0x10cc6a290>
>>> client.collection("status")
<google.cloud.firestore_v1beta1.collection.CollectionReference object at 0x10e8c5f50>
>>> col = client.collection("status")
>>> doc = col.document(14)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/collection.py", line 107, in document
    return self._client.document(*child_path)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/client.py", line 214, in document
    return DocumentReference(*path, client=self)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/document.py", line 52, in __init__
    _helpers.verify_path(path, is_collection=False)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 370, in verify_path
    raise ValueError(msg)
ValueError: A path element must be a string. Received 14, which is a <type 'int'>.
>>> doc = col.document("14")
>>> doc
<google.cloud.firestore_v1beta1.document.DocumentReference object at 0x10e8d8110>
>>> firestore.__version__
'0.29.0'
>>> doc.update({14: {'status': 'active'}})
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/document.py", line 370, in update
    batch.update(self, field_updates, option=option)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/batch.py", line 101, in update
    self._client, reference._document_path, field_updates, option)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 891, in pbs_for_update
    update_values, field_paths = FieldPathHelper.to_field_paths(actual_updates)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 325, in to_field_paths
    return helper.parse()
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 303, in parse
    self.add_value_at_field_path(key, value)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 281, in add_value_at_field_path
    parts = parse_field_path(field_path)
  File "/Users/jdubinsky/projects/firestore-th/firestore_th/firestore_th/lib/python2.7/site-packages/google/cloud/firestore_v1beta1/_helpers.py", line 599, in parse_field_path
    return field_path.split(FIELD_PATH_DELIMITER)
AttributeError: 'int' object has no attribute 'split'
>>> doc.update({'14': {'status': 'active'}})
update_time {
  seconds: 1536348609
  nanos: 596332000
}