Last time a DB was modified?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Last time a DB was modified?

Ron Johnson
Hi,

I've got a cron job that backs up databases every night, whether they've
been updated or even viewed.  Naturally, that's a big waste of space.

The timestamp fields of ~/.gramps/recent-files-gramps.xml are a good start
for selectively backing up databases, but they record whenever the DB is
opened, regardless of whether any data has been modified.

So: is there any CLI way of determining if a db has been modified? Maybe
peeking directly at the bdb with some tool or a python script?

Thanks

--
My word, man!  Don't you know your quantum statistics?


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Last time a DB was modified?

Sebastian Schubert
Hi Ron,

> I've got a cron job that backs up databases every night, whether they've
> been updated or even viewed.  Naturally, that's a big waste of space.

I'd like to propose another strategy for you backup based on an a
combination of XML export and a version control system: You can export
your catalogue into a Gramps XML file. This file includes the same data
as your catalogue (please correct if I am wrong). A version control
system can then be used to save only the changes. In addition, you can
evaluate yourself which changes were done.

In my case, everytime I do some serious editing, I convert the database
into xml (script below), check in the changes with git and do a git push
to save the changes on another server. (In my case, all my media files
are also included in the repository, making it easy to just check out
the git repository on another machine.) So far, this worked perfectly
for me.

Here is my script to export the db to xml:

###############################
FTREE="MYTREE"
REPO="${HOME}/gramps/"
GRAMPSDB="${HOME}/.gramps/grampsdb/"

# export family tree and extract it
TMPFILE=`mktemp`
gramps -y -e "${TMPFILE}" -f gramps -O "${FTREE}"
gunzip < "${TMPFILE}" > "${REPO}/${FTREE}.gramps"
rm "${TMPFILE}"
###############################

FTREE is the name of my tree and REPO is my git repository. As a result,
${FTREE}.gramps includes your data and can be processed with git (or any
other version control).

In principle, loading the xml file back into gramps is easy. However,
last time I checked, just running it with Gramps imports the data as
additional information. This way you end up with the data twice. This
can be solved by deleting the database of the tree before. Since I did
not find a way to get the correct specific path of my tree and I just
have one tree, I just delete the whole grampsdb folder:

###############################
FTREE="MYTREE"
REPO="${HOME}/gramps/"
GRAMPSDB="${HOME}/.gramps/grampsdb/"

# remove binary gramps database files
rm -r "${GRAMPSDB}"

# open family tree for editing
gramps "${REPO}/${FTREE}.gramps"
###############################

Maybe this helps,
Sebastian

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Last time a DB was modified?

Ron Johnson
In reply to this post by Ron Johnson
On 06/12/2014 07:52 AM, Sebastian Schubert wrote:

> Hi Ron,
>
>> I've got a cron job that backs up databases every night, whether they've
>> been updated or even viewed.  Naturally, that's a big waste of space.
> I'd like to propose another strategy for you backup based on an a
> combination of XML export and a version control system: You can export
> your catalogue into a Gramps XML file. This file includes the same data
> as your catalogue (please correct if I am wrong). A version control
> system can then be used to save only the changes. In addition, you can
> evaluate yourself which changes were done.
>
> In my case, everytime I do some serious editing, I convert the database
> into xml (script below), check in the changes with git and do a git push
> to save the changes on another server. (In my case, all my media files
> are also included in the repository, making it easy to just check out
> the git repository on another machine.) So far, this worked perfectly
> for me.

That's a clever idea!

--
My word, man!  Don't you know your quantum statistics?


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Last time a DB was modified?

Peter (chamdo4ever)
I'm the height of ignorance speaking here, to be sure....

I'm a big fan of the sync storage service SpiderOak
(www.spideroak.com) -- what if I were to have SpiderOak pointed to my
.gramps folder and automatically sync and backup any changes made to
my database that way?

Beyond just backing up, might that also be a way to seamlessly
maintain my database (and Gramps configurations) on more than one
machine?


Peter

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Last time a DB was modified?

jerome
In reply to this post by Sebastian Schubert
Hi,


For fun, I ran commands on a Gramps XML file, using lxml for some performances issues, and came to the conclusion that we can quickly generate a "squeleton" without major privacy issues! eg, on the second level (XML hierarchy), to store items for each elements will generate set of keys/values, which might be markers for revisions control too. ;)

This did not have sense into current gramps environment, and it was performed via command lines: just an other script. On the other hand, this is a proof that the XML scheme/model is consistent and that current timestamp on primary objects could be used for filtering modified objects, making runtime faster. :)

Today, I rather use 'diff' tool on gunziped .gramps files.
eg, $ diff -u old.gramps new.gramps > changes.patch

Otherwise, Doug seems to start to provide something more interesting[1] on next major release[2] !



Jérôme

Le jeu. 12 juin 2014 at 14:52, Sebastian Schubert <[hidden email]> a écrit :
Hi Ron,
I've got a cron job that backs up databases every night, whether they've been updated or even viewed. Naturally, that's a big waste of space.
I'd like to propose another strategy for you backup based on an a combination of XML export and a version control system: You can export your catalogue into a Gramps XML file. This file includes the same data as your catalogue (please correct if I am wrong). A version control system can then be used to save only the changes. In addition, you can evaluate yourself which changes were done. In my case, everytime I do some serious editing, I convert the database into xml (script below), check in the changes with git and do a git push to save the changes on another server. (In my case, all my media files are also included in the repository, making it easy to just check out the git repository on another machine.) So far, this worked perfectly for me. Here is my script to export the db to xml: ############################### FTREE="MYTREE" REPO="${HOME}/gramps/" GRAMPSDB="${HOME}/.gramps/grampsdb/" # export family tree and extract it TMPFILE=`mktemp` gramps -y -e "${TMPFILE}" -f gramps -O "${FTREE}" gunzip < "${TMPFILE}" > "${REPO}/${FTREE}.gramps" rm "${TMPFILE}" ############################### FTREE is the name of my tree and REPO is my git repository. As a result, ${FTREE}.gramps includes your data and can be processed with git (or any other version control). In principle, loading the xml file back into gramps is easy. However, last time I checked, just running it with Gramps imports the data as additional information. This way you end up with the data twice. This can be solved by deleting the database of the tree before. Since I did not find a way to get the correct specific path of my tree and I just have one tree, I just delete the whole grampsdb folder: ############################### FTREE="MYTREE" REPO="${HOME}/gramps/" GRAMPSDB="${HOME}/.gramps/grampsdb/" # remove binary gramps database files rm -r "${GRAMPSDB}" # open family tree for editing gramps "${REPO}/${FTREE}.gramps" ############################### Maybe this helps, Sebastian ------------------------------------------------------------------------------ HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions Find What Matters Most in Your Big Data with HPCC Systems Open Source. Fast. Scalable. Simple. Ideal for Dirty Data. Leverages Graph Analysis for Fast Processing & Easy Data Exploration http://p.sf.net/sfu/hpccsystems _______________________________________________ Gramps-users mailing list [hidden email] https://lists.sourceforge.net/lists/listinfo/gramps-users

------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Specific tree path (was Re: Last time a DB was modified?)

Ron Johnson
In reply to this post by Ron Johnson
On 06/12/2014 07:52 AM, Sebastian Schubert wrote:
[snip]
> In principle, loading the xml file back into gramps is easy. However,
> last time I checked, just running it with Gramps imports the data as
> additional information. This way you end up with the data twice. This
> can be solved by deleting the database of the tree before. Since I did
> not find a way to get the correct specific path of my tree and I just
> have one tree, I just delete the whole grampsdb folder:

$ gramps -l
List of known Family Trees in your database path

/home/ron/.gramps/grampsdb/5397eebc with name "Collins-Costa"
/home/ron/.gramps/grampsdb/5397bdc8 with name "Gilbert-Bakewell"
/home/ron/.gramps/grampsdb/5397eeb0 with name "Johnson-Lyle"
/home/ron/.gramps/grampsdb/5398e426 with name "Testing"

--
My word, man!  Don't you know your quantum statistics?


------------------------------------------------------------------------------
HPCC Systems Open Source Big Data Platform from LexisNexis Risk Solutions
Find What Matters Most in Your Big Data with HPCC Systems
Open Source. Fast. Scalable. Simple. Ideal for Dirty Data.
Leverages Graph Analysis for Fast Processing & Easy Data Exploration
http://p.sf.net/sfu/hpccsystems
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users