Using OpenRefine to clean up Gramps data

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Using OpenRefine to clean up Gramps data

GRAMPS - User mailing list
I have misspelling, duplication and inconsistency in my Gramps tree. I
have been trying to use OpenRefine to correct these.

I backed up and exported the XML to OpenRefine successfully, and used
its tools to correct a few errors.

Now I would like to export from OpenRefine the cleaned up data in the
same, XML, format as I imported it, so as to import it back into
Gramps.  I can't see how to do this. Is it possible?

Does anyone have experience of using OpenRefine to clean up Gramps data?

David Lynch




--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Using OpenRefine to clean up Gramps data

Ron Johnson
On 11/2/19 12:30 PM, David Lynch via Gramps-users wrote:

> I have misspelling, duplication and inconsistency in my Gramps tree. I
> have been trying to use OpenRefine to correct these.
>
> I backed up and exported the XML to OpenRefine successfully, and used its
> tools to correct a few errors.
>
> Now I would like to export from OpenRefine the cleaned up data in the
> same, XML, format as I imported it, so as to import it back into Gramps. 
> I can't see how to do this. Is it possible?
>
> Does anyone have experience of using OpenRefine to clean up Gramps data?
>
> David Lynch

Shouldn't this be asked on the OpenRefine group?
https://groups.google.com/forum/#!forum/openrefine


--
Angular momentum makes the world go 'round.


--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Using OpenRefine to clean up Gramps data

GRAMPS - User mailing list
On 02/11/2019 18:04, Ron Johnson wrote:

> On 11/2/19 12:30 PM, David Lynch via Gramps-users wrote:
>> I have misspelling, duplication and inconsistency in my Gramps tree.
>> I have been trying to use OpenRefine to correct these.
>>
>> I backed up and exported the XML to OpenRefine successfully, and used
>> its tools to correct a few errors.
>>
>> Now I would like to export from OpenRefine the cleaned up data in the
>> same, XML, format as I imported it, so as to import it back into
>> Gramps.  I can't see how to do this. Is it possible?
>>
>> Does anyone have experience of using OpenRefine to clean up Gramps data?
>>
>> David Lynch
>
> Shouldn't this be asked on the OpenRefine group?
> https://groups.google.com/forum/#!forum/openrefine
>
I tried (10 days ago) on that group, with no reply.

However, since I sent this (less than an hour ago) there has been a
reply. It looks complicated and I will need a template for the Gramps
XML. So, if anyone has done it for Gramps, I'd be grateful for their wisdom.

David Lynch



--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Using OpenRefine to clean up Gramps data

Connie Mack

I've done some clean up with Open Refine for gramps but I exported as csv, did the work and exported from Open Refine as csvs too. Since it was my first time doing this, I created a test copy of my tree and imported the csv file into that. It worked well. Open Refine is a great tool. 


On Sat, Nov 2, 2019 at 2:13 PM David Lynch via Gramps-users <[hidden email]> wrote:
On 02/11/2019 18:04, Ron Johnson wrote:
> On 11/2/19 12:30 PM, David Lynch via Gramps-users wrote:
>> I have misspelling, duplication and inconsistency in my Gramps tree.
>> I have been trying to use OpenRefine to correct these.
>>
>> I backed up and exported the XML to OpenRefine successfully, and used
>> its tools to correct a few errors.
>>
>> Now I would like to export from OpenRefine the cleaned up data in the
>> same, XML, format as I imported it, so as to import it back into
>> Gramps.  I can't see how to do this. Is it possible?
>>
>> Does anyone have experience of using OpenRefine to clean up Gramps data?
>>
>> David Lynch
>
> Shouldn't this be asked on the OpenRefine group?
> https://groups.google.com/forum/#!forum/openrefine
>
I tried (10 days ago) on that group, with no reply.

However, since I sent this (less than an hour ago) there has been a
reply. It looks complicated and I will need a template for the Gramps
XML. So, if anyone has done it for Gramps, I'd be grateful for their wisdom.

David Lynch



--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org


--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Using OpenRefine to clean up Gramps data

Ron Johnson
In reply to this post by GRAMPS - User mailing list
On 11/2/19 1:13 PM, David Lynch via Gramps-users wrote:

> On 02/11/2019 18:04, Ron Johnson wrote:
>> On 11/2/19 12:30 PM, David Lynch via Gramps-users wrote:
>>> I have misspelling, duplication and inconsistency in my Gramps tree. I
>>> have been trying to use OpenRefine to correct these.
>>>
>>> I backed up and exported the XML to OpenRefine successfully, and used
>>> its tools to correct a few errors.
>>>
>>> Now I would like to export from OpenRefine the cleaned up data in the
>>> same, XML, format as I imported it, so as to import it back into
>>> Gramps.  I can't see how to do this. Is it possible?
>>>
>>> Does anyone have experience of using OpenRefine to clean up Gramps data?
>>>
>>> David Lynch
>>
>> Shouldn't this be asked on the OpenRefine group?
>> https://groups.google.com/forum/#!forum/openrefine
>>
> I tried (10 days ago) on that group, with no reply.
>
> However, since I sent this (less than an hour ago) there has been a reply.
> It looks complicated and I will need a template for the Gramps XML. So, if
> anyone has done it for Gramps, I'd be grateful for their wisdom.

Looking at the top of a decompressed .gramps file, I see:

<!DOCTYPE database PUBLIC "-//Gramps//DTD Gramps XML 1.7.1//EN"
"http://gramps-project.org/xml/1.7.1/grampsxml.dtd">
<database xmlns="http://gramps-project.org/xml/1.7.1/">

You can download the .dtd file.  Maybe it can help can help you make the yaml.


--
Angular momentum makes the world go 'round.


--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Using OpenRefine to clean up Gramps data

Nick Hall
On 02/11/2019 18:58, Ron Johnson wrote:
> You can download the .dtd file.  Maybe it can help can help you make
> the yaml.

A RelaxNG schema is also available:

https://github.com/gramps-project/gramps/blob/maintenance/gramps51/data/grampsxml.rng


Nick.




--
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org