Upcoming pickle protocol change

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Upcoming pickle protocol change

GRAMPS - Dev mailing list
Dear Gramps devs,

sorry if this question has been discussed before or if I should have filed a bug instead.

I noticed that the upcoming Python 3.8, to be released in a few weeks, will change the default protocol version used by pickle from 3 (around since Python 3.0) to 4 (introduced in Python 3.4).
As far as I can see, at the moment the pickling of blobs in Gramps does not specify the protocol. I believe this will lead to backwards incompatibility if someone edits a database with Gramps on Python 3.8 and then later tries to open it on Python 3.3. This might happen e.g. for people using the same database on different computers.

A very simple solution to this potential problem would be to just pin the protocol version to the current default, i.e. add protocol=3 to all occurences of pickle.dumps in the source. This is backward and forward compatible and wouldn't change anything about the current behaviour.

This might become obsolete when Python 3.4 is made the minimum required version or JSON in used instead of blobs (I don't know what the status of  that plan is), but for the moment (i.e. for the 5.1.x series) I believe it would make sense to pin the protocol to avoid any troubles with 3.8.

If you agree, I would be happy to prepare a PR.

Cheers,
David





_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Upcoming pickle protocol change

Nick Hall
On 04/09/2019 17:45, David Straub via Gramps-devel wrote:
> sorry if this question has been discussed before or if I should have
> filed a bug instead.
>
> I noticed that the upcoming Python 3.8, to be released in a few weeks,
> will change the default protocol version used by pickle from 3 (around
> since Python 3.0) to 4 (introduced in Python 3.4).
> Source: https://docs.python.org/3.8/whatsnew/3.8.html


I was unaware of this change.  Thanks for pointing it out.


> As far as I can see, at the moment the pickling of blobs in Gramps
> does not specify the protocol. I believe this will lead to backwards
> incompatibility if someone edits a database with Gramps on Python 3.8
> and then later tries to open it on Python 3.3. This might happen e.g.
> for people using the same database on different computers.
>
> A very simple solution to this potential problem would be to just pin
> the protocol version to the current default, i.e. add protocol=3 to
> all occurences of pickle.dumps in the source. This is backward and
> forward compatible and wouldn't change anything about the current
> behaviour.


This sounds like a sensible solution, but is it necessary? Python 3.3
and possibly also 3.4 have already reached their end-of-life.  Is any
distribution still using them?


>
> This might become obsolete when Python 3.4 is made the minimum
> required version or JSON in used instead of blobs (I don't know what
> the status of  that plan is), but for the moment (i.e. for the 5.1.x
> series) I believe it would make sense to pin the protocol to avoid any
> troubles with 3.8.
>
> If you agree, I would be happy to prepare a PR.
>

Does anyone have an objection to this suggestion?


Nick.




_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel