Filters to delete descendants in V5.

classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

Filters to delete descendants in V5.

Peter Merchant

I am using Gramps 5.0.0 in kubunto 18.04 and trying to create a filter to delete all descendants of a person. All I can find is under the edit menu a 'person filter editor'. Is this what I need? It doesn't follow the wiki documentation very well.

Here is what I am trying to do:  [any comments welcome]

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.


In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.


Filter: Section 16 of the manual.

https://gramps-project.org/wiki/index.php/Gramps_5.0_Wiki_Manual_-_Filters


https://gramps-project.org/wiki/index.php?title=Example_filters#Example_3._All_female_descendants_of_Joan_Doe


Thanks for any help.

Peter M.



_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

enno

Hello Peter,

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.

In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.

To me, this sounds way more complicated than what I did, after I inherited my late father's work, which was 4 trees in Brother's Keeper, based on my grandparents. I imported those in PAF, that is before I learned about Gramps, and let its duplicate tool do the hard work.

The only difference between my situation and yours is that you have 8 trees, which will obviously give more duplicates, but unless you have hundreds of descendants, I don't think that's a problem that's too big for you to handle.

With that in mind, I suggest that you create ONE new database, import the others into that, and start merging. You can do a lot of merging by hand, by starting with yourself and your wife in the person list view, selecting 2 copies, and clicking the merge button. And if you have the relatives Gramplet open, you will immediately see the effect in the sense that a merged person will have duplicate relatives, and you can click in that Gramplet to navigate to one relative, and merge that relative with his or her copy, and so on, and so on. I prefer to do that in a recursive fashion, till I have weeded out all obvious duplicates.

I prefer this over the more complicated pruning in your plan, because it will immediately show inconsistencies in what you think are true duplicates. Such inconsistencies are easy to create when you had the habit to add each descendant manually in all 8 trees up till now, because you might have forgotten an event, or two. I found such inconsistencies in my father's work, so that's why I mention this.

Note that, when you merge persons, Gramps does not merge associated events or notes, so most merged persons will have 2 births etc., immediately after merge, and if you merge first, and take out the weed later, they will have 8, same for notes. And merging events is quite a nuisance, because it can't be done in the person editor, I think, at least not in my old Gramps 3.4. And this is also true for family events, meaning that a merged couple will have multiple marriage events too.

If you have unknown consistencies, and I bet you have, this is the safest method, unless you're absolutely sure that one database has the best data for your duplicates.

When I find duplicate events I often start the person or family editor, and simply delete all duplicates, while making sure that I never delete an event that has notes or sources that another instance of it doesn't have. If I find one, I switch to genuine merging in the event view, which will still mean that you'll have duplicate citations too.

Note that, even when you've removed all duplicate events from families, duplicate relationships will still exist. You need the relationship view to weed out those. In that, you can disconnect any person from a duplicate relationship, meaning a marriage, or his/her duplicate parent couple.

When you delete duplicate events, you'll end up with orphan ones, which may have associated places that can become orphans too, once the events are removed. You can clean those with a tool too, the one that finds disconnected objects. You have to run that a couple of times, because a place is not an orphan as long as it's associated with an event, so it will only find them if you let it remove the orphaned events first.

Once done with this, you can also run the find duplicates tool to find any surprise relationships. :-)

I hope this helps a bit. I think that it's a lot safer than filters, because you'll never loose anything, unless you choose to delete it yourself.

Regards,

Enno




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Peter Merchant
On 28/12/2018 22:28, Enno Borgsteede wrote:

Hello Peter,

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.

In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.

To me, this sounds way more complicated than what I did, after I inherited my late father's work, which was 4 trees in Brother's Keeper, based on my grandparents. I imported those in PAF, that is before I learned about Gramps, and let its duplicate tool do the hard work.

The only difference between my situation and yours is that you have 8 trees, which will obviously give more duplicates, but unless you have hundreds of descendants, I don't think that's a problem that's too big for you to handle.

With that in mind, I suggest that you create ONE new database, import the others into that, and start merging. You can do a lot of merging by hand, by starting with yourself and your wife in the person list view, selecting 2 copies, and clicking the merge button. And if you have the relatives Gramplet open, you will immediately see the effect in the sense that a merged person will have duplicate relatives, and you can click in that Gramplet to navigate to one relative, and merge that relative with his or her copy, and so on, and so on. I prefer to do that in a recursive fashion, till I have weeded out all obvious duplicates.

I prefer this over the more complicated pruning in your plan, because it will immediately show inconsistencies in what you think are true duplicates. Such inconsistencies are easy to create when you had the habit to add each descendant manually in all 8 trees up till now, because you might have forgotten an event, or two. I found such inconsistencies in my father's work, so that's why I mention this.

Note that, when you merge persons, Gramps does not merge associated events or notes, so most merged persons will have 2 births etc., immediately after merge, and if you merge first, and take out the weed later, they will have 8, same for notes. And merging events is quite a nuisance, because it can't be done in the person editor, I think, at least not in my old Gramps 3.4. And this is also true for family events, meaning that a merged couple will have multiple marriage events too.

If you have unknown consistencies, and I bet you have, this is the safest method, unless you're absolutely sure that one database has the best data for your duplicates.

When I find duplicate events I often start the person or family editor, and simply delete all duplicates, while making sure that I never delete an event that has notes or sources that another instance of it doesn't have. If I find one, I switch to genuine merging in the event view, which will still mean that you'll have duplicate citations too.

Note that, even when you've removed all duplicate events from families, duplicate relationships will still exist. You need the relationship view to weed out those. In that, you can disconnect any person from a duplicate relationship, meaning a marriage, or his/her duplicate parent couple.

When you delete duplicate events, you'll end up with orphan ones, which may have associated places that can become orphans too, once the events are removed. You can clean those with a tool too, the one that finds disconnected objects. You have to run that a couple of times, because a place is not an orphan as long as it's associated with an event, so it will only find them if you let it remove the orphaned events first.

Once done with this, you can also run the find duplicates tool to find any surprise relationships. :-)

I hope this helps a bit. I think that it's a lot safer than filters, because you'll never loose anything, unless you choose to delete it yourself.


Thanks Enno, very sensible advice.

If I edit this can I add it to the wiki as a suggested method of combining databases?  Once I have done the work that is.

Peter



_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Peter Merchant
On 29/12/2018 11:48, Peter Merchant wrote:
On 28/12/2018 22:28, Enno Borgsteede wrote:

Hello Peter,

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.

In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.

To me, this sounds way more complicated than what I did, after I inherited my late father's work, which was 4 trees in Brother's Keeper, based on my grandparents. I imported those in PAF, that is before I learned about Gramps, and let its duplicate tool do the hard work.

The only difference between my situation and yours is that you have 8 trees, which will obviously give more duplicates, but unless you have hundreds of descendants, I don't think that's a problem that's too big for you to handle.

With that in mind, I suggest that you create ONE new database, import the others into that, and start merging. You can do a lot of merging by hand, by starting with yourself and your wife in the person list view, selecting 2 copies, and clicking the merge button. And if you have the relatives Gramplet open, you will immediately see the effect in the sense that a merged person will have duplicate relatives, and you can click in that Gramplet to navigate to one relative, and merge that relative with his or her copy, and so on, and so on. I prefer to do that in a recursive fashion, till I have weeded out all obvious duplicates.

I prefer this over the more complicated pruning in your plan, because it will immediately show inconsistencies in what you think are true duplicates. Such inconsistencies are easy to create when you had the habit to add each descendant manually in all 8 trees up till now, because you might have forgotten an event, or two. I found such inconsistencies in my father's work, so that's why I mention this.

Note that, when you merge persons, Gramps does not merge associated events or notes, so most merged persons will have 2 births etc., immediately after merge, and if you merge first, and take out the weed later, they will have 8, same for notes. And merging events is quite a nuisance, because it can't be done in the person editor, I think, at least not in my old Gramps 3.4. And this is also true for family events, meaning that a merged couple will have multiple marriage events too.

If you have unknown consistencies, and I bet you have, this is the safest method, unless you're absolutely sure that one database has the best data for your duplicates.

When I find duplicate events I often start the person or family editor, and simply delete all duplicates, while making sure that I never delete an event that has notes or sources that another instance of it doesn't have. If I find one, I switch to genuine merging in the event view, which will still mean that you'll have duplicate citations too.

Note that, even when you've removed all duplicate events from families, duplicate relationships will still exist. You need the relationship view to weed out those. In that, you can disconnect any person from a duplicate relationship, meaning a marriage, or his/her duplicate parent couple.

When you delete duplicate events, you'll end up with orphan ones, which may have associated places that can become orphans too, once the events are removed. You can clean those with a tool too, the one that finds disconnected objects. You have to run that a couple of times, because a place is not an orphan as long as it's associated with an event, so it will only find them if you let it remove the orphaned events first.

Once done with this, you can also run the find duplicates tool to find any surprise relationships. :-)

I hope this helps a bit. I think that it's a lot safer than filters, because you'll never loose anything, unless you choose to delete it yourself.


Thanks Enno, very sensible advice.

If I edit this can I add it to the wiki as a suggested method of combining databases?  Once I have done the work that is.

Peter

Another question: Would it be easier to add the other databases to my master one one at a time and work through the duplicates, or to add them all at once and then do it, in your opinion?

Peter



_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

enno
Hello Peter,

> On 29/12/2018 11:48, Peter Merchant wrote:
>> If I edit this can I add it to the wiki as a suggested method of
>> combining databases?  Once I have done the work that is.
>>
>> Peter
>>
> Another question: Would it be easier to add the other databases to my
> master one one at a time and work through the duplicates, or to add
> them all at once and then do it, in your opinion?
>
> Peter
>
Since I really like to work in a state of flow, I'm inclined to import
all at once, and work through the duplicates later. The latter may take
several days to complete, while you're learning what works best for you.

And once you know, please add it to the wiki, adapting text where needed.

Cheers,

Enno




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Doug-11
In reply to this post by Peter Merchant
On 29/12/2018 12:08, Peter Merchant wrote:
On 29/12/2018 11:48, Peter Merchant wrote:
On 28/12/2018 22:28, Enno Borgsteede wrote:

Hello Peter,

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.

In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.

To me, this sounds way more complicated than what I did, after I inherited my late father's work, which was 4 trees in Brother's Keeper, based on my grandparents. I imported those in PAF, that is before I learned about Gramps, and let its duplicate tool do the hard work.

The only difference between my situation and yours is that you have 8 trees, which will obviously give more duplicates, but unless you have hundreds of descendants, I don't think that's a problem that's too big for you to handle.

With that in mind, I suggest that you create ONE new database, import the others into that, and start merging. You can do a lot of merging by hand, by starting with yourself and your wife in the person list view, selecting 2 copies, and clicking the merge button. And if you have the relatives Gramplet open, you will immediately see the effect in the sense that a merged person will have duplicate relatives, and you can click in that Gramplet to navigate to one relative, and merge that relative with his or her copy, and so on, and so on. I prefer to do that in a recursive fashion, till I have weeded out all obvious duplicates.

I prefer this over the more complicated pruning in your plan, because it will immediately show inconsistencies in what you think are true duplicates. Such inconsistencies are easy to create when you had the habit to add each descendant manually in all 8 trees up till now, because you might have forgotten an event, or two. I found such inconsistencies in my father's work, so that's why I mention this.

Note that, when you merge persons, Gramps does not merge associated events or notes, so most merged persons will have 2 births etc., immediately after merge, and if you merge first, and take out the weed later, they will have 8, same for notes. And merging events is quite a nuisance, because it can't be done in the person editor, I think, at least not in my old Gramps 3.4. And this is also true for family events, meaning that a merged couple will have multiple marriage events too.

If you have unknown consistencies, and I bet you have, this is the safest method, unless you're absolutely sure that one database has the best data for your duplicates.

When I find duplicate events I often start the person or family editor, and simply delete all duplicates, while making sure that I never delete an event that has notes or sources that another instance of it doesn't have. If I find one, I switch to genuine merging in the event view, which will still mean that you'll have duplicate citations too.

Note that, even when you've removed all duplicate events from families, duplicate relationships will still exist. You need the relationship view to weed out those. In that, you can disconnect any person from a duplicate relationship, meaning a marriage, or his/her duplicate parent couple.

When you delete duplicate events, you'll end up with orphan ones, which may have associated places that can become orphans too, once the events are removed. You can clean those with a tool too, the one that finds disconnected objects. You have to run that a couple of times, because a place is not an orphan as long as it's associated with an event, so it will only find them if you let it remove the orphaned events first.

Once done with this, you can also run the find duplicates tool to find any surprise relationships. :-)

I hope this helps a bit. I think that it's a lot safer than filters, because you'll never loose anything, unless you choose to delete it yourself.


Thanks Enno, very sensible advice.

If I edit this can I add it to the wiki as a suggested method of combining databases?  Once I have done the work that is.

Peter

Another question: Would it be easier to add the other databases to my master one one at a time and work through the duplicates, or to add them all at once and then do it, in your opinion?

Peter


I don't know off-hand a way to lighten your task with filters, bearing in mind Enno's warning. But I'd strongly recommend you do the adding databases one at a time. Sorting out the proper relationship trees when you have them partially merged over several generations can be a bit of a nightmare - even when there are only two to deal with.

I'm still counting my bruises!

Doug





_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

enno
Op 29-12-18 om 15:05 schreef Doug:
> I don't know off-hand a way to lighten your task with filters, bearing
> in mind Enno's warning. But I'd strongly recommend you do the adding
> databases one at a time. Sorting out the proper relationship trees
> when you have them partially merged over several generations can be a
> bit of a nightmare - even when there are only two to deal with.
>
> I'm still counting my bruises!
>
Ah, good to know. I understand that it may be easier to learn the
procedure, and the find duplicate tool will also run faster.

Regards,

Enno




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Lionel Haslam
In reply to this post by Doug-11
Hi Peter,

I recently merged 4 separate trees into one for my brother in law. My approach was to find the matched people in each tree and make sure the details matched. Fortunately there  was a max of about 10 people overlapping in each tree. I made a plan for each tree before starting. Then I removed the duplicates bar one common strategic person in the trees before I started to merge.
It turned out to be easy this way as I knew where the link was going to be and didn't need to search for matching people afterwards. (Gramps makes this approach very easy as you can use the "Not Related" add on before joining the trees to identify the separate trees before the merge.)
Making a plan is the most important step in my opinion. (Of course if the trees have large numbers of random duplicates this approach may not work.)

Lionel

From: Doug [hidden email]
Sent: Saturday, December 29, 2018 4:05PM
To: Peter Merchant [hidden email], Gramps-users [hidden email]
Subject: Re: [Gramps-users] Filters to delete descendants in V5.

 


On 29/12/2018 12:08, Peter Merchant wrote:
On 29/12/2018 11:48, Peter Merchant wrote:
On 28/12/2018 22:28, Enno Borgsteede wrote:

Hello Peter,

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.

In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.

To me, this sounds way more complicated than what I did, after I inherited my late father's work, which was 4 trees in Brother's Keeper, based on my grandparents. I imported those in PAF, that is before I learned about Gramps, and let its duplicate tool do the hard work.

The only difference between my situation and yours is that you have 8 trees, which will obviously give more duplicates, but unless you have hundreds of descendants, I don't think that's a problem that's too big for you to handle.

With that in mind, I suggest that you create ONE new database, import the others into that, and start merging. You can do a lot of merging by hand, by starting with yourself and your wife in the person list view, selecting 2 copies, and clicking the merge button. And if you have the relatives Gramplet open, you will immediately see the effect in the sense that a merged person will have duplicate relatives, and you can click in that Gramplet to navigate to one relative, and merge that relative with his or her copy, and so on, and so on. I prefer to do that in a recursive fashion, till I have weeded out all obvious duplicates.

I prefer this over the more complicated pruning in your plan, because it will immediately show inconsistencies in what you think are true duplicates. Such inconsistencies are easy to create when you had the habit to add each descendant manually in all 8 trees up till now, because you might have forgotten an event, or two. I found such inconsistencies in my father's work, so that's why I mention this.

Note that, when you merge persons, Gramps does not merge associated events or notes, so most merged persons will have 2 births etc., immediately after merge, and if you merge first, and take out the weed later, they will have 8, same for notes. And merging events is quite a nuisance, because it can't be done in the person editor, I think, at least not in my old Gramps 3.4. And this is also true for family events, meaning that a merged couple will have multiple marriage events too.

If you have unknown consistencies, and I bet you have, this is the safest method, unless you're absolutely sure that one database has the best data for your duplicates.

When I find duplicate events I often start the person or family editor, and simply delete all duplicates, while making sure that I never delete an event that has notes or sources that another instance of it doesn't have. If I find one, I switch to genuine merging in the event view, which will still mean that you'll have duplicate citations too.

Note that, even when you've removed all duplicate events from families, duplicate relationships will still exist. You need the relationship view to weed out those. In that, you can disconnect any person from a duplicate relationship, meaning a marriage, or his/her duplicate parent couple.

When you delete duplicate events, you'll end up with orphan ones, which may have associated places that can become orphans too, once the events are removed. You can clean those with a tool too, the one that finds disconnected objects. You have to run that a couple of times, because a place is not an orphan as long as it's associated with an event, so it will only find them if you let it remove the orphaned events first.

Once done with this, you can also run the find duplicates tool to find any surprise relationships. :-)

I hope this helps a bit. I think that it's a lot safer than filters, because you'll never loose anything, unless you choose to delete it yourself.


Thanks Enno, very sensible advice.

If I edit this can I add it to the wiki as a suggested method of combining databases?  Once I have done the work that is.

Peter

Another question: Would it be easier to add the other databases to my master one one at a time and work through the duplicates, or to add them all at once and then do it, in your opinion?

Peter


I don't know off-hand a way to lighten your task with filters, bearing in mind Enno's warning. But I'd strongly recommend you do the adding databases one at a time. Sorting out the proper relationship trees when you have them partially merged over several generations can be a bit of a nightmare - even when there are only two to deal with.

I'm still counting my bruises!

Doug





_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org


_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Doug-11
On 29/12/2018 16:13, Lionel Haslam wrote:
Hi Peter,

I recently merged 4 separate trees into one for my brother in law. My approach was to find the matched people in each tree and make sure the details matched. Fortunately there  was a max of about 10 people overlapping in each tree. I made a plan for each tree before starting. Then I removed the duplicates bar one common strategic person in the trees before I started to merge.
It turned out to be easy this way as I knew where the link was going to be and didn't need to search for matching people afterwards. (Gramps makes this approach very easy as you can use the "Not Related" add on before joining the trees to identify the separate trees before the merge.)
Making a plan is the most important step in my opinion. (Of course if the trees have large numbers of random duplicates this approach may not work.)

Lionel

From: Doug [hidden email]
Sent: Saturday, December 29, 2018 4:05PM
To: Peter Merchant [hidden email], Gramps-users [hidden email]
Subject: Re: [Gramps-users] Filters to delete descendants in V5.

 


On 29/12/2018 12:08, Peter Merchant wrote:
On 29/12/2018 11:48, Peter Merchant wrote:
On 28/12/2018 22:28, Enno Borgsteede wrote:

Hello Peter,

The accepted way of working in Gramps is to have a single large database. My situation is that as I started working I created a database for each grandparents families. That is four databases for my ancestors and four for my wife’s. Let us call them families a,b,c,d...h. In each of these databases myself, my wife and my descendants are duplicated, which is a lot of work adding each new grandchild. Each one of these families comes from a different part of the country so there is not much duplication of places, except for descendants.

In order to combine these into one I am going to do the following steps.

1. Back up each database.

2. Create databases A-1, B-1, C-1 etc and restore the appropriate backup into each of these

3. leaving aside A-1, for each of the others apply a filter that deletes my parents and their descendants. This is described later.

4. for each of these do any tidying up that is obvious, and then do a backup.

5. restore these backups into A-1, the new master database.

6. Is there a search for duplicate names? If so merge any duplicates, and do the same for places.

To me, this sounds way more complicated than what I did, after I inherited my late father's work, which was 4 trees in Brother's Keeper, based on my grandparents. I imported those in PAF, that is before I learned about Gramps, and let its duplicate tool do the hard work.

The only difference between my situation and yours is that you have 8 trees, which will obviously give more duplicates, but unless you have hundreds of descendants, I don't think that's a problem that's too big for you to handle.

With that in mind, I suggest that you create ONE new database, import the others into that, and start merging. You can do a lot of merging by hand, by starting with yourself and your wife in the person list view, selecting 2 copies, and clicking the merge button. And if you have the relatives Gramplet open, you will immediately see the effect in the sense that a merged person will have duplicate relatives, and you can click in that Gramplet to navigate to one relative, and merge that relative with his or her copy, and so on, and so on. I prefer to do that in a recursive fashion, till I have weeded out all obvious duplicates.

I prefer this over the more complicated pruning in your plan, because it will immediately show inconsistencies in what you think are true duplicates. Such inconsistencies are easy to create when you had the habit to add each descendant manually in all 8 trees up till now, because you might have forgotten an event, or two. I found such inconsistencies in my father's work, so that's why I mention this.

Note that, when you merge persons, Gramps does not merge associated events or notes, so most merged persons will have 2 births etc., immediately after merge, and if you merge first, and take out the weed later, they will have 8, same for notes. And merging events is quite a nuisance, because it can't be done in the person editor, I think, at least not in my old Gramps 3.4. And this is also true for family events, meaning that a merged couple will have multiple marriage events too.

If you have unknown consistencies, and I bet you have, this is the safest method, unless you're absolutely sure that one database has the best data for your duplicates.

When I find duplicate events I often start the person or family editor, and simply delete all duplicates, while making sure that I never delete an event that has notes or sources that another instance of it doesn't have. If I find one, I switch to genuine merging in the event view, which will still mean that you'll have duplicate citations too.

Note that, even when you've removed all duplicate events from families, duplicate relationships will still exist. You need the relationship view to weed out those. In that, you can disconnect any person from a duplicate relationship, meaning a marriage, or his/her duplicate parent couple.

When you delete duplicate events, you'll end up with orphan ones, which may have associated places that can become orphans too, once the events are removed. You can clean those with a tool too, the one that finds disconnected objects. You have to run that a couple of times, because a place is not an orphan as long as it's associated with an event, so it will only find them if you let it remove the orphaned events first.

Once done with this, you can also run the find duplicates tool to find any surprise relationships. :-)

I hope this helps a bit. I think that it's a lot safer than filters, because you'll never loose anything, unless you choose to delete it yourself.


Thanks Enno, very sensible advice.

If I edit this can I add it to the wiki as a suggested method of combining databases?  Once I have done the work that is.

Peter

Another question: Would it be easier to add the other databases to my master one one at a time and work through the duplicates, or to add them all at once and then do it, in your opinion?

Peter


I don't know off-hand a way to lighten your task with filters, bearing in mind Enno's warning. But I'd strongly recommend you do the adding databases one at a time. Sorting out the proper relationship trees when you have them partially merged over several generations can be a bit of a nightmare - even when there are only two to deal with.

I'm still counting my bruises!

Doug

From what Lionel says I think it probably depends on how much overlap and duplication you have to deal with. It's true I've had little trouble in merging databases where there's limited intersection; on the other hand the tree I received from my grandfather's brother's grandson which intersected widely with my existing tree caused a great deal of grief. RMMV.

Doug




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Peter Merchant
On 29/12/2018 18:10, Doug wrote:
On 29/12/2018 16:13, Lionel Haslam wrote:
Hi Peter,

I recently merged 4 separate trees into one for my brother in law. My approach was to find the matched people in each tree and make sure the details matched. Fortunately there  was a max of about 10 people overlapping in each tree. I made a plan for each tree before starting. Then I removed the duplicates bar one common strategic person in the trees before I started to merge.
..........

I'm still counting my bruises!

Doug

From what Lionel says I think it probably depends on how much overlap and duplication you have to deal with. It's true I've had little trouble in merging databases where there's limited intersection; on the other hand the tree I received from my grandfather's brother's grandson which intersected widely with my existing tree caused a great deal of grief. RMMV.

Doug

I had a simple plan. For the next one it will be more detailed. I went through my paternal Grandmother tree and deleted quite a few people who I knew to be common to both trees. There were already some of Grandmothers ancestors in my master tree which I left.

Luckily after importing the  cut-down tree into my master Gramps gave me some import statistics that show 921 records that can be merged (921 isn't lucky). I pasted these into a spreadsheet that I can  mark as I do them. 

For the next family that I want to import I will cut back more people.

Lessons learned! For places it is probably easiest to leave them to be merged later than trying to compare beforehand. Events also, unless it is possible to see how to delete events associated with deleted families.


I'll go quiet now while I do 921 merges.    Peter




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

enno
Hi Peter,

> I had a simple plan. For the next one it will be more detailed. I went
> through my paternal Grandmother tree and deleted quite a few people
> who I knew to be common to both trees. There were already some of
> Grandmothers ancestors in my master tree which I left.
And you knew that the deleted persons had no more data then their
counterparts? Just asking ...
>
> Luckily after importing the  cut-down tree into my master Gramps gave
> me some import statistics that show 921 records that can be merged
> (921 isn't lucky). I pasted these into a spreadsheet that I can  mark
> as I do them.
>
As far as I know, this only means that the imported file had 921
records, whatever that means. In my mind, Gramps' import statistics are
a bit more specific than that. Do you mean 921 persons?
>
> For the next family that I want to import I will cut back more people.
>
Ok, if you're sure that they're all identical, meaning that they don't
have more data than their counterparts.
>
> Lessons learned! For places it is probably easiest to leave them to be
> merged later than trying to compare beforehand. Events also, unless it
> is possible to see how to delete events associated with deleted families.
>
If you run the remove unconnected objects tool, or whatever that's
called in English, it will find all orphan events, meaning events that
are not referenced by a person or family. And once you remove those, and
run the tool again, it will also find places that were orphaned by the
previous run. I use this every time after some import, and it works very
well.

I support the idea of merging places afterwards. I wrote a tool for
Gramps 3.4 that merges all places when they have the same title, and no
place details, but I haven't thought of such a thing for Gramps 4.2 and
newer. My places are still too messy for that.

> I'll go quiet now while I do 921 merges.
>
I bet it's just a handful, maybe a dozen. You got fooled by the import
statistics, which have no idea about the real number of duplicates.

Cheers,

Enno




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Peter Merchant
On 30/12/2018 13:09, Enno Borgsteede wrote:
Hi Peter,

I had a simple plan. For the next one it will be more detailed. I went through my paternal Grandmother tree and deleted quite a few people who I knew to be common to both trees. There were already some of Grandmothers ancestors in my master tree which I left.
And you knew that the deleted persons had no more data then their counterparts? Just asking ...
Yes, checked before deleting.


If you run the remove unconnected objects tool, or whatever that's called in English, it will find all orphan events, meaning events that are not referenced by a person or family. And once you remove those, and run the tool again, it will also find places that were orphaned by the previous run. I use this every time after some import, and it works very well.
Thought about that afterwards. will include it in next plan.


I bet it's just a handful, maybe a dozen. You got fooled by the import statistics, which have no idea about the real number of duplicates.


The import statistics said:"

Objects that are candidates to be merged:

and so far it's proving to be true.

P.



_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

prculley
In reply to this post by enno
The new Gramps 5.0.0 Place cleanup tool can help you with merging places.

Fyi

Paul Culley 

On Sun, Dec 30, 2018, 7:11 AM Enno Borgsteede <[hidden email] wrote:
Hi Peter,

> I had a simple plan. For the next one it will be more detailed. I went
> through my paternal Grandmother tree and deleted quite a few people
> who I knew to be common to both trees. There were already some of
> Grandmothers ancestors in my master tree which I left.
And you knew that the deleted persons had no more data then their
counterparts? Just asking ...
>
> Luckily after importing the  cut-down tree into my master Gramps gave
> me some import statistics that show 921 records that can be merged
> (921 isn't lucky). I pasted these into a spreadsheet that I can  mark
> as I do them.
>
As far as I know, this only means that the imported file had 921
records, whatever that means. In my mind, Gramps' import statistics are
a bit more specific than that. Do you mean 921 persons?
>
> For the next family that I want to import I will cut back more people.
>
Ok, if you're sure that they're all identical, meaning that they don't
have more data than their counterparts.
>
> Lessons learned! For places it is probably easiest to leave them to be
> merged later than trying to compare beforehand. Events also, unless it
> is possible to see how to delete events associated with deleted families.
>
If you run the remove unconnected objects tool, or whatever that's
called in English, it will find all orphan events, meaning events that
are not referenced by a person or family. And once you remove those, and
run the tool again, it will also find places that were orphaned by the
previous run. I use this every time after some import, and it works very
well.

I support the idea of merging places afterwards. I wrote a tool for
Gramps 3.4 that merges all places when they have the same title, and no
place details, but I haven't thought of such a thing for Gramps 4.2 and
newer. My places are still too messy for that.

> I'll go quiet now while I do 921 merges.
>
I bet it's just a handful, maybe a dozen. You got fooled by the import
statistics, which have no idea about the real number of duplicates.

Cheers,

Enno




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org


_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Doug-11
In reply to this post by Peter Merchant
On 30/12/2018 13:49, Peter Merchant wrote:
On 30/12/2018 13:09, Enno Borgsteede wrote:
Hi Peter,

I had a simple plan. For the next one it will be more detailed. I went through my paternal Grandmother tree and deleted quite a few people who I knew to be common to both trees. There were already some of Grandmothers ancestors in my master tree which I left.
And you knew that the deleted persons had no more data then their counterparts? Just asking ...
Yes, checked before deleting.


If you run the remove unconnected objects tool, or whatever that's called in English, it will find all orphan events, meaning events that are not referenced by a person or family. And once you remove those, and run the tool again, it will also find places that were orphaned by the previous run. I use this every time after some import, and it works very well.
Thought about that afterwards. will include it in next plan.


I bet it's just a handful, maybe a dozen. You got fooled by the import statistics, which have no idea about the real number of duplicates.


The import statistics said:"

Objects that are candidates to be merged:

and so far it's proving to be true.

P.


A couple of additional tools - useful in reconciling relationship trees with imports from elsewhere:

Sort children in birth order

Sorts events

Doug




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filters to delete descendants in V5.

Doug-11
On 30/12/2018 19:40, Doug wrote:
On 30/12/2018 13:49, Peter Merchant wrote:
On 30/12/2018 13:09, Enno Borgsteede wrote:
Hi Peter,

I had a simple plan. For the next one it will be more detailed. I went through my paternal Grandmother tree and deleted quite a few people who I knew to be common to both trees. There were already some of Grandmothers ancestors in my master tree which I left.
And you knew that the deleted persons had no more data then their counterparts? Just asking ...
Yes, checked before deleting.


If you run the remove unconnected objects tool, or whatever that's called in English, it will find all orphan events, meaning events that are not referenced by a person or family. And once you remove those, and run the tool again, it will also find places that were orphaned by the previous run. I use this every time after some import, and it works very well.
Thought about that afterwards. will include it in next plan.


I bet it's just a handful, maybe a dozen. You got fooled by the import statistics, which have no idea about the real number of duplicates.


The import statistics said:"

Objects that are candidates to be merged:

and so far it's proving to be true.

P.


A couple of additional tools - useful in reconciling relationship trees with imports from elsewhere:

Sort children in birth order

Sorts events

Doug


Another tip: it can sometimes be helpful to compare directly the tree, events list, etc in the database you're creating with the one you're using as source (for import, for corrections, etc). To do it in Linux Mint, use the multiple workspaces facility.

Open gramps on one workspace and load the first family tree; then switch to another workspace, open gramps again and load the other family tree.

You can toggle between one workspace and the other. You can work on either tree in its own workspace. Each tree is locked outside its own workspace.

HTH


Doug




_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org