struct/JSON in master

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

struct/JSON in master

DS Blank
Devs,

FYI, I've been working on an alternative representation for objects
for a while. I've called this a "struct" in the code, as it is a
combination of dictionaries, lists, and values, much like JSON. You
can think of these as an annotated serialized form of data.

Before, every gen.lib object could be converted using obj.to_struct().
Now, there is are also from_struct() class methods on all of the
gen.lib objects that take structs as input, and give back a
gramps-serialized form of the object. Some example uses (from the
Python Gramplet):

>>> p = db.get_person_from_gramps_id("I0001")
>>> p.to_struct()
{u'parent_family_list': ['48TJQCGNNIR5SJRCAK'], u'private': False,
u'event_ref_list': [{'attribute_list': [], 'private': False, 'role':
{u'_class': 'EventRoleType', u'string': 'Primary', u'value': 1},
'_class': 'EventRef', 'ref': 'a5af0eb667015e355db', ... }
>>> serialized = Person.from_struct(p.to_struct())
>>> p2 = Person.create(serialized)

These structs are used in the addon Database Differences Report to
find any differences between them:

>>> from gramps.gen.merge.diff import diff_items
>>> diff = diff_items("Person", p.to_struct(), p2.to_struct())

Next, my goal is to make it so that we can make "patches" to sync a
gramps-connect website and a gramps gtk app. So, this struct can be
used to pass info back and forth. Should be easier to use than XML, or
the raw serialized version. Struct data also includes some metadata,
so they can be versioned, etc.

Also, I'm updating a SQL-like query and updating interface. You can
see the start of this with this example, where you give a dotted path:

>>> from gramps.gen.merge.diff import Struct
>>> s = Struct(p.to_struct(), db)
>>> s["primary_name.surname_list.0.surname"]
Smith
>>> s["primary_name.surname_list.0.surname"] = "Jones"
>>> s["primary_name.surname_list.0.surname"]
Jones

That is, you can get (and set) a part of an object by its path (keys
in dicts and positions in lists), and it updates the database.

I'll put some of this detail on a wiki page to document various uses,
and updates as it progresses.

-Doug

------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: struct/JSON in master

DS Blank
Oh, one other part: there is the beginning of a JSON import/export
plugin in gramps-addons/trunk. The code is very simple because the
data is just dictionaries (one record per line in file). It loads
large databases very quickly because Python does the parsing as the
JSON/dictionary representation can just be evaled.

This could be a fairly easy alternative to moving data in and out of
gramps. Also, if you don't specify a field, it uses whatever is the
default:

>>> from gramps.gen.merge.diff import from_struct
>>> from_struct( {"_class": "Person", "primary_name": {"first_name": "Sample"}})
<Person>

-Doug

On Sun, Nov 10, 2013 at 5:02 PM, Doug Blank <[hidden email]> wrote:

> Devs,
>
> FYI, I've been working on an alternative representation for objects
> for a while. I've called this a "struct" in the code, as it is a
> combination of dictionaries, lists, and values, much like JSON. You
> can think of these as an annotated serialized form of data.
>
> Before, every gen.lib object could be converted using obj.to_struct().
> Now, there is are also from_struct() class methods on all of the
> gen.lib objects that take structs as input, and give back a
> gramps-serialized form of the object. Some example uses (from the
> Python Gramplet):
>
>>>> p = db.get_person_from_gramps_id("I0001")
>>>> p.to_struct()
> {u'parent_family_list': ['48TJQCGNNIR5SJRCAK'], u'private': False,
> u'event_ref_list': [{'attribute_list': [], 'private': False, 'role':
> {u'_class': 'EventRoleType', u'string': 'Primary', u'value': 1},
> '_class': 'EventRef', 'ref': 'a5af0eb667015e355db', ... }
>>>> serialized = Person.from_struct(p.to_struct())
>>>> p2 = Person.create(serialized)
>
> These structs are used in the addon Database Differences Report to
> find any differences between them:
>
>>>> from gramps.gen.merge.diff import diff_items
>>>> diff = diff_items("Person", p.to_struct(), p2.to_struct())
>
> Next, my goal is to make it so that we can make "patches" to sync a
> gramps-connect website and a gramps gtk app. So, this struct can be
> used to pass info back and forth. Should be easier to use than XML, or
> the raw serialized version. Struct data also includes some metadata,
> so they can be versioned, etc.
>
> Also, I'm updating a SQL-like query and updating interface. You can
> see the start of this with this example, where you give a dotted path:
>
>>>> from gramps.gen.merge.diff import Struct
>>>> s = Struct(p.to_struct(), db)
>>>> s["primary_name.surname_list.0.surname"]
> Smith
>>>> s["primary_name.surname_list.0.surname"] = "Jones"
>>>> s["primary_name.surname_list.0.surname"]
> Jones
>
> That is, you can get (and set) a part of an object by its path (keys
> in dicts and positions in lists), and it updates the database.
>
> I'll put some of this detail on a wiki page to document various uses,
> and updates as it progresses.
>
> -Doug

------------------------------------------------------------------------------
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel