Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

Mike Toogood
I deleted my working db in error. I tried to import my most recent backup, which includes media, gpkg file.

It failed. I tried opening the file with archive manager in linux - that failed too.
I found that 4 of the backed up media files had strange first characters in
their names ( ? and ! ). I did this by reinstating backups (i back up daily)
until restoring the backup failed and Renaming the bad names and reloading a
backup dummy into a new database.

The most recent backup that works is three months old. I have nearly 2000 media objects.

There was no warning about bad media file names when Gramps created the .gpkg back
ups.

Gramps cannot partially restore the back up without restoring media.
My question is can I get the .gramps DB data out of the .gpkg backup file without
trying to extract the backed up media and then reconstruct my most recently
backed up database?
I am ignorant of which command line commands might be used to unpack just the .gramps file from the .gpkg archive outside of gramps.
Any help gratefully received. I put a lot of work in on my db in the last three months!
Moral of this story  *always confirm your back ups can be imported* !

Mike Toogood




------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

Ron Johnson
1) What version are you running?
2) The failure error message would be very helpful.
3) Did you try restoring from the command line?  (I doubt it, but had to ask.)
4) What is the exact name of the gpkg file?

Try these commands from a command window:
a) gzip --verbose --test "your-latest-backup.gpkg"
b) gramps -C Foo
c) gramps -O Foo  --import "your-latest-backup.gpkg"




On 02/26/2015 04:38 AM, Mike Toogood wrote:
I deleted my working db in error. I tried to import my most recent backup, which includes media, gpkg file.

It failed. I tried opening the file with archive manager in linux - that failed too.
I found that 4 of the backed up media files had strange first characters in
their names ( ? and ! ). I did this by reinstating backups (i back up daily)
until restoring the backup failed and Renaming the bad names and reloading a
backup dummy into a new database.

The most recent backup that works is three months old. I have nearly 2000 media objects.

There was no warning about bad media file names when Gramps created the .gpkg back
ups.

Gramps cannot partially restore the back up without restoring media.
My question is can I get the .gramps DB data out of the .gpkg backup file without
trying to extract the backed up media and then reconstruct my most recently
backed up database?
I am ignorant of which command line commands might be used to unpack just the .gramps file from the .gpkg archive outside of gramps.
Any help gratefully received. I put a lot of work in on my db in the last three months!
Moral of this story  *always confirm your back ups can be imported* !

-- 
My word, man!  Don't you know your quantum statistics?

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

enno
In reply to this post by Mike Toogood
Hello Mike,
I deleted my working db in error. I tried to import my most recent backup, which includes media, gpkg file.

It failed. I tried opening the file with archive manager in linux - that failed too.
I found that 4 of the backed up media files had strange first characters in
their names ( ? and ! ). I did this by reinstating backups (i back up daily)
until restoring the backup failed and Renaming the bad names and reloading a
backup dummy into a new database.
Can you put the bad backup on Dropbox, Google Drive, or OneDrive? I like to see what Gramps says here, when it tries to read the file. You can use my email address to share the file with me. I hope that it's not like 1 GB in size, but even if it is, I may be able to download it.
The most recent backup that works is three months old. I have nearly 2000 media objects.

There was no warning about bad media file names when Gramps created the .gpkg back
ups.
H'm, so you have 'bad' names on your hard drive then, right? Just curious.
Gramps cannot partially restore the back up without restoring media.
My question is can I get the .gramps DB data out of the .gpkg backup file without
trying to extract the backed up media and then reconstruct my most recently
backed up database?
With normal backups, i.e. non corrupted ones, you can. Here's how:
1. Rename the .gpkg to .gz, and unpack that with the archive manager,
2. Rename the extracted .gpkg to .tar, and extract the data.gramps file from that.

You may try to extract all files from the .tar, but I assume that when you deleted your DB, you left the media files alone, so then it makes no sense to extract those again.

If the archive manager fails on the renamed files, you may find help on-line for corrupted .gz or .tar.
I am ignorant of which command line commands might be used to unpack just the .gramps file from the .gpkg archive outside of gramps.
See above. I did all that with the GUI.
Any help gratefully received. I put a lot of work in on my db in the last three months!
Moral of this story  *always confirm your back ups can be imported* !

      
My motto: Use .gramps backups for your DB, and backup media in another way. They normally don't change that much, so you can sync them with an external drive, put them in the cloud, whatever you like. This works much faster and because of smaller backup sizes, you can save way more on-line.

regards,

Enno


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

Mike Toogood
In reply to this post by Ron Johnson
Thanks everyone.

I have managed to recover the .gramps file from the .gpkg archive.

I followed instructions above provided by enno.

I was able to get back the .gramps file using Archive Manager, linux, Debian Stable, by selecting only the .gramps file to extract

I have implemented a new back up strategy - no media backup from within Gramps - and check back up by re-importing.

I run:

GRAMPS: 3.4.8-1
Python: 2.7.2 (default, Jun 12 2011, 15:08:59) [...
BSDDB: 4.8.4.1 (4, 7, 25)
LANG: en_GB.UTF-8
OS: win32

Mike
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

enno
Mike,
> Thanks everyone.
>
> I have managed to recover the .gramps file from the .gpkg archive.
>
> I followed instructions above provided by enno.
>
> I was able to get back the .gramps file using Archive Manager, linux, Debian
> Stable, by selecting only the .gramps file to extract
That's great. And reading that you normally use Windows, I think that
7-Zip can do the job too, maybe even without renaming files, if you use
the context menu to open them.
> I have implemented a new back up strategy - no media backup from within
> Gramps - and check back up by re-importing.
Good idea. But since you mentioned it before, it makes sense to see
whether Gramps itself can be made a bit more robust with respect to
weird names in files and paths. Can you make a bug report for that, and
maybe attach a small test .gpkg with odd media names?

thanks,

Enno


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

Peter (chamdo4ever)
In reply to this post by enno
Hi Enno,

On Thu, Feb 26, 2015 at 6:49 AM, Enno Borgsteede <[hidden email]> wrote:
> My motto: Use .gramps backups for your DB, and backup media in another way.
> They normally don't change that much, so you can sync them with an external
> drive, put them in the cloud, whatever you like. This works much faster and
> because of smaller backup sizes, you can save way more on-line.

So, you just backup your Gramps DB without media and then if you do a
restore, you point the Gramps Media Manager to the media folder on
your hard drive?

Thanks,

Peter

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Recovering .gramps from .gpkg. Original problem probably low disk space.

Mike Toogood
In reply to this post by Mike Toogood
Following Enno's suggestion above I have tried to reproduce the problem by making a dummy gramps DB with only the media files I suspected archived in the .gpkg backup.

I could not reproduce the problem.

The strange characters mentioned previously at the beginning of some file names arrived when I had cut and pasted some text to use as a media file name including tab and eol control characters. I now believe that they were not the cause of my decompression error.

My gpkg file is about 2gigabytes.  Double decompression might need at least 6 Gig of HD space for temp files and new decompressed file. I had about 4 gig free. Not enough disk space.

It would be helpful if the error warning given by gramps on unsuccessful .gpkg import was a bit more specific - at least stating whether there was a corrupt archive or low disk space.

I have used Gramps for about seven years. I really like it.

I am glad it was my fault and not Gramps'!

I used to run Gramps on Debian. But I migrated it to Windows XP because I wanted the extra features that are in the current Win XP version as I run the latest Debian Stable and have always used the Gramps version distributed by Debian, which is of course quite mature compared to the most recent release on Sourceforge. I have to run XP because I also use my laptop on my Sailboat and the software for my ancient boat's NAVTEX radio hardware is only written for XP.
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg. Original problem probably low disk space.

enno
Hi Mike,
> My gpkg file is about 2gigabytes.  Double decompression might need at least
> 6 Gig of HD space for temp files and new decompressed file. I had about 4
> gig free. Not enough disk space.
>
> It would be helpful if the error warning given by gramps on unsuccessful
> .gpkg import was a bit more specific - at least stating whether there was a
> corrupt archive or low disk space.
Can you make a bug report for that? And assuming that the big file is
still there, can you check whether you get any usable error messages
when you Gramps from console, or terminal in Debian?
> I am glad it was my fault and not Gramps'!
Me too, but it also shows that working with large archives is a risky
business. Maybe we should advise .gramps as the standard format for
backup, and inform fellow users to backup media in the usual way.

I'm not sure whether OneDrive still works with Windows XP, but I bet
Dropbox does, and I think that you can better let those tools do the
media backups for you, or use something like FreeFileSync (provided that
you avoid the adware during install).

regards,

Enno


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

enno
In reply to this post by Peter (chamdo4ever)
Hi Peter,
> So, you just backup your Gramps DB without media and then if you do a
> restore, you point the Gramps Media Manager to the media folder on
> your hard drive?
Sort of. I have never actually gone through the restore procedure, but
when I do, it's probably even simpler than that, because the media path
is set in preferences, and either saved in an INI file, which doesn't
get lost that fast, or in the DB itself, and gets restored when I
restore the backup.

I use FreeFileSync to backup my full home directory (except a couple of
system files) to an external drive, and sync the Documents and Pictures
directories on that with Windows too, with the same tool. And in
Windows, these folders sit in a OneDrive folder, so they're also in the
cloud.

If you're on Linux only, there are of course simpler ways, but I need
Windows for a couple of things, and free storage on Dropbox is not large
enough for me.

regards,

Enno


------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
Reply | Threaded
Open this post in threaded view
|

Re: Recovering .gramps from .gpkg file. Can I recover .gramps file, without the media files?

Peter (chamdo4ever)
Thank you Enno -- the below all makes good sense.

Peter

On Fri, Feb 27, 2015 at 9:20 AM, Enno Borgsteede <[hidden email]> wrote:

> Hi Peter,
>>
>> So, you just backup your Gramps DB without media and then if you do a
>> restore, you point the Gramps Media Manager to the media folder on your hard
>> drive?
>
> Sort of. I have never actually gone through the restore procedure, but when
> I do, it's probably even simpler than that, because the media path is set in
> preferences, and either saved in an INI file, which doesn't get lost that
> fast, or in the DB itself, and gets restored when I restore the backup.
>
> I use FreeFileSync to backup my full home directory (except a couple of
> system files) to an external drive, and sync the Documents and Pictures
> directories on that with Windows too, with the same tool. And in Windows,
> these folders sit in a OneDrive folder, so they're also in the cloud.
>
> If you're on Linux only, there are of course simpler ways, but I need
> Windows for a couple of things, and free storage on Dropbox is not large
> enough for me.
>
> regards,
>
> Enno
>

------------------------------------------------------------------------------
Dive into the World of Parallel Programming The Go Parallel Website, sponsored
by Intel and developed in partnership with Slashdot Media, is your hub for all
things parallel software development, from weekly thought leadership blogs to
news, videos, case studies, tutorials and more. Take a look and join the
conversation now. http://goparallel.sourceforge.net/
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users