Filtering hint for merging in Gramps

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Filtering hint for merging in Gramps

GRAMPS - User mailing list
I'm trying to develop techniques for merging duplicate families in Gramps.

What I've come up with uses Tags & Filters. But the technique needs other people to critique and suggest ideas.

There are 2 specific cases:

1) importing a family grouping sent as a GEDcom or XML.  There is SOME duplication of the existing anchoring parents and partial list of offspring. (By default, the importing process adds a timestamp Tag to the records of imported People, Family, Media, & Notes. These potential duplicates can be spotted easily in a Name-Sorted list by changing to a non-black color for this import Tag... use Light Grey for these not-yet-validated records.)  

2) Where I've already transcribed a Branch and then discovered the "unknown last name" bride entered elsewhere is a spouse in this branch.

Both cases use the same set of Tag & Filter techniques.

First, locate and confirm that person has a true duplicate. Select and Merge the duplicate persons. Add a new Orange Tag named "DupeFilter" to the central merged person.

Here's where Gramps (like every other Genealogy database) gets ugly...

If there are duplicate event records & relationship for the merged person, all those have to be individually merged. Unfortunately,there isn't an easy way to limit the Events or People lists to those associated with a particular person.

And if the Relationship shows duplicate parents, spouses, sibs or children; there will be a MUCH more complex task of keeping track of all those people and merging all of THEIR records and relationships.

You COULD build a Parent+Sib+Spouse+Offspring filter for a specific person to do this. But then the filter rules would have to be redefined every time you had to merge a different pair of duplicates. Bookmarking has some other limits that make it unsuitable for the task.

But a more dynamically changing filter can be created using Tags. However, since some filters cannot used Tagged People as a criteria, there's a little extra setup. Most importantly, the rules for finding Events or Parents/Sibs/Spouse/Offpring related persons can't find Tagged people directly.

Fortunately, you only have to set up the rules once. They are flexible and re-useable.

Normally, you find Tagged People by just selecting it from the Tag pop-up menu in the Filter gramplet. The following is the equivalent using Filter rules. It uses that orange colored "DupeFilter" tag we created above.

Now in People, go to the Filter gramplet in the Sidebar and Edit the 'None' Custom Filter to add a People filter named "DupeFilterPeople"
Add the General filters rule to find 'People with the <tag> "DupeFilter"' (you can ignore the Options since there is only a single rule in the filter.)

We can use this "DupeFilterPeople" filter when building a custom filter for Events related to our Tagged person.

In the Filter gramplet for Events, create a Custom filter called "DupeFilterPeopleEvents".  Add the General filters rule 'Events of persons matching the <person filter> "DupeFilterPeople"'
Select the checkbox to "Inclde Family events"
Again, the other Options for "Define filter" are not important.

Now when you "Find" with this filter, only the Events related to the Tagged person appear.

You could manually tag the Siblings, Parents, Spouses, Offspring; and then use the filter again. But that would be error prone.

Instead, let's add a new filter to show the people related to our Tagged person, mass-Tag them and re-Find associated Events. The added benefit is that this will simplify merging the duplicate immediate relatives.

Create another custom filter in the Filter gramplet for People called "DupeFilterRelations"
Add 4 Family filters rules:
  'Parents of <filter> match "DupeFilterPeople"'
  'Siblings of <filter> match "DupeFilterPeople"'
  'Spouses of <filter> match "DupeFilterPeople"'
  'Children of <filter> match "DupeFilterPeople"'
In Options, 'At least one rule must apply'

After 'Find'ing thes relations, do an extended select of everyone found and add the "DupeFilter" to them all. (This can take a few seconds for a large tree.) Reset the Filter gramplet and just use the Tag pop-up filter menu to Find "DupeFilter again. It will make column sorts much more responsive than the 'Custom filter' as you Merge duplicate immediate family members.

Re-Find in the Events to clean up the duplicate immediate family events. Use the same Tagging of all the found Events to minimize time using the custom filter.

If you re-run the People Custom filter after Tagging the immediate family, that will find great-grandparents, in-law parents, grandchildren, and relatives from spouse's other marriages.

One caveat, it is easy to mass-add Tags to a group of people yet they can only be removed 1 by 1 or from the ENTIRE tree.  

So, keep this in mind and plan to periodically Delete the "DupeFinder" tag using the Organize Tags feature.. The rules won't need to be redefined even when the Tag is destroyed IF recreate a fresh Tag with the identical name.

These Filters REALLY streamline cleaning up a duplicate family. Please try them! Then tell me where I skipped a step or missed an alternative.

Brian

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filtering hint for merging in Gramps

Doug-11
[hidden email] <[hidden email]> wrote:

I'm trying to develop techniques for merging duplicate families in Gramps.

What I've come up with uses Tags & Filters. But the technique needs other people to critique and suggest ideas.

There are 2 specific cases:

1) importing a family grouping sent as a GEDcom or XML.  There is SOME duplication of the existing anchoring parents and partial list of offspring. (By default, the importing process adds a timestamp Tag to the records of imported People, Family, Media, & Notes. These potential duplicates can be spotted easily in a Name-Sorted list by changing to a non-black color for this import Tag... use Light Grey for these not-yet-validated records.)

2) Where I've already transcribed a Branch and then discovered the "unknown last name" bride entered elsewhere is a spouse in this branch.

Both cases use the same set of Tag & Filter techniques.

First, locate and confirm that person has a true duplicate. Select and Merge the duplicate persons. Add a new Orange Tag named "DupeFilter" to the central merged person.

Here's where Gramps (like every other Genealogy database) gets ugly...

If there are duplicate event records & relationship for the merged person, all those have to be individually merged. Unfortunately,there isn't an easy way to limit the Events or People lists to those associated with a particular person.

And if the Relationship shows duplicate parents, spouses, sibs or children; there will be a MUCH more complex task of keeping track of all those people and merging all of THEIR records and relationships.

You COULD build a Parent+Sib+Spouse+Offspring filter for a specific person to do this. But then the filter rules would have to be redefined every time you had to merge a different pair of duplicates. Bookmarking has some other limits that make it unsuitable for the task.

But a more dynamically changing filter can be created using Tags. However, since some filters cannot used Tagged People as a criteria, there's a little extra setup. Most importantly, the rules for finding Events or Parents/Sibs/Spouse/Offpring related persons can't find Tagged people directly.

Fortunately, you only have to set up the rules once. They are flexible and re-useable.

Normally, you find Tagged People by just selecting it from the Tag pop-up menu in the Filter gramplet. The following is the equivalent using Filter rules. It uses that orange colored "DupeFilter" tag we created above.

Now in People, go to the Filter gramplet in the Sidebar and Edit the 'None' Custom Filter to add a People filter named "DupeFilterPeople"
Add the General filters rule to find 'People with the <tag> "DupeFilter"' (you can ignore the Options since there is only a single rule in the filter.)

We can use this "DupeFilterPeople" filter when building a custom filter for Events related to our Tagged person.

In the Filter gramplet for Events, create a Custom filter called "DupeFilterPeopleEvents".  Add the General filters rule 'Events of persons matching the <person filter> "DupeFilterPeople"'
Select the checkbox to "Inclde Family events"
Again, the other Options for "Define filter" are not important.

Now when you "Find" with this filter, only the Events related to the Tagged person appear.

You could manually tag the Siblings, Parents, Spouses, Offspring; and then use the filter again. But that would be error prone.

Instead, let's add a new filter to show the people related to our Tagged person, mass-Tag them and re-Find associated Events. The added benefit is that this will simplify merging the duplicate immediate relatives.

Create another custom filter in the Filter gramplet for People called "DupeFilterRelations"
Add 4 Family filters rules:
   'Parents of <filter> match "DupeFilterPeople"'
   'Siblings of <filter> match "DupeFilterPeople"'
   'Spouses of <filter> match "DupeFilterPeople"'
   'Children of <filter> match "DupeFilterPeople"'
In Options, 'At least one rule must apply'

After 'Find'ing thes relations, do an extended select of everyone found and add the "DupeFilter" to them all. (This can take a few seconds for a large tree.) Reset the Filter gramplet and just use the Tag pop-up filter menu to Find "DupeFilter again. It will make column sorts much more responsive than the 'Custom filter' as you Merge duplicate immediate family members.

Re-Find in the Events to clean up the duplicate immediate family events. Use the same Tagging of all the found Events to minimize time using the custom filter.

If you re-run the People Custom filter after Tagging the immediate family, that will find great-grandparents, in-law parents, grandchildren, and relatives from spouse's other marriages.

One caveat, it is easy to mass-add Tags to a group of people yet they can only be removed 1 by 1 or from the ENTIRE tree.

So, keep this in mind and plan to periodically Delete the "DupeFinder" tag using the Organize Tags feature.. The rules won't need to be redefined even when the Tag is destroyed IF recreate a fresh Tag with the identical name.

These Filters REALLY streamline cleaning up a duplicate family. Please try them! Then tell me where I skipped a step or missed an alternative.

Brian


Why don't you tidy this up and add it to the Gramps Wiki Example filters? It'll surely be useful.
https://gramps-project.org/wiki/index.php?title=Example_filters

Doug


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filtering hint for merging in Gramps

GRAMPS - User mailing list
In reply to this post by GRAMPS - User mailing list
Thanks Doug. Adding to the documentation Wiki is one of my goals.

Keyword suggestions are also welcome. Beside Filter examples, I thought about seeing what keywords are on the Merging docs too.

Currently have over a dozen screen captures and need to explore the wiki's illustration/capture filenaming convention so that other language & OS captures will be easier to generate. (Maybe I can find equivalent captures already in the library and avoid wasting server space on dupes.)

But I need some help proof reading the basic text for confusing steps, errors & omissions. Can't do that effectively on your own tech docs.

Brian


--------------------------------------------
On Sun, 7/15/18, Doug <[hidden email]> wrote:

 Subject: Re: [Gramps-users] Filtering hint for merging in Gramps
 Date: Sunday, July 15, 2018, 4:25 PM
 
 Why don't you tidy this up and add it
 to the Gramps Wiki Example filters? It'll surely be useful.
 https://gramps-project.org/wiki/index.php?title=Example_filters
 
 Doug
 
 

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org