Family ID - How Determined?

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Family ID - How Determined?

tcorbet
I have been using Gramps for some time and currently have three versions
installed: 3.4.9, 4.2.5 and 5.0.0.  I am mentioning that mostly just to
answer the question as to what version am I talking about.  The
difficulty I have just noticed occurs the same in all three cases.

My workflow is to download the latest version of my gedcom file from
Ancestry.com and import it into Gramps.  Since the maintenance of all my
source data is on Ancestry, I delete the current Gramps database and
import into a newly-created one.

The rest of my workflow involves an export of the Gramps database to a
sqlite format where I use various SQL statements to produce text output
which is then processed to accomplish some of my reporting goals.  This
has all been working really well for about five years, but yesterday the
process broke due to data conditions which I do not understand.  I hope
someone can point me to a brief description of how the assignment of
Family IDs is performed by the import loader.

Let me clarify that when I say I perform a gedcom import, I mean that I
simply follow the sequence of prompts [slightly different for different
versions of the Gramps software, but essentially the same across all
three versions] allowing the 'auto' feature to identify the type of
input from the selection of the MyTree.ged file.  There are no other
controls or settings that I am aware of or am applying.

What it has been my happy experience has been that the two key
table/views created by Gramps, Persons and Families, obtain their IDs
from the gedcom tags INDI and FAMS.  When I say that is a happy result,
I mostly mean that it is quite convenient to have a common set of
identifiers that can be relied upon when working in the two different
environments.  But, if Gramps had had some other means of generating the
unique keys that it requires, I would have not objected, I would simply
have written slightly-different SQL to take into account the two
different ways of doing things.

So, what all of a sudden is failing in my logic is that, for the first
time that I have noticed, some of the ID keys in the Family table/view
in Gramps have no equivalents in the imported gedcom file.  Sorry for
the long introduction, but I expect that most of that context will be
required to help explain the answer as to: "Under what circumstances
does the Gramps import/loader generate a new Family ID rather than use
the one contained in the incoming gedcom file?"

I regularly look at all of the diagnostic messages which flow from the
import/load, and am familiar with the reasons that Gramps drops or
otherwise notices incoming content that it does not like.  There are no
diagnostic notices that might explain why some FAMS tag might have been
found to be invalid and some substitution to have been made in order,
for example, to preserve the necessary relational integrity as between
Persons and Families.

I am not stating that I think Gramps logic is incorrect, indeed, since
the new circumstance only showed up after performing changes to the
source gedcom at Ancestry, the most likely cause of the new and
unexpected behavior, is on the incoming data side of things. Obviously
on a continuing basis, additions and deletions of Persons and Families,
causes the Ancestry database maintenance code to sometimes dynamically
re-assign its own keys.  So, as between point-in-time #One and
point-in-time #Two I could observe that the Ancestry.com-assigned ID for
an individual would change. That is expected behavior and works fine as
long as the relational integrity between tables/views remains correct.  
So, whatever my most recent additions and deletions have been, with
whatever changes of the identifiers within Ancestry have been, that
system remains consistent AND after the import, Gramps remain
consistent.  The strangeness is only in respect of not understanding why
Gramps has generated a handful of Family identifiers not supplied by the
Ancestry source.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Family ID - How Determined?

enno
Hello Terry,

> donSo, what all of a sudden is failing in my logic is that, for the first
> time that I have noticed, some of the ID keys in the Family table/view
> in Gramps have no equivalents in the imported gedcom file.  Sorry for
> the long introduction, but I expect that most of that context will be
> required to help explain the answer as to: "Under what circumstances
> does the Gramps import/loader generate a new Family ID rather than use
> the one contained in the incoming gedcom file?"
The only one that I can think of is when Gramps discovers that some ID
is not unique. And since you always import into an empty tree, this
could only happen when there are duplicates in the Ancestry GEDCOM, or a
clash with IDs generated by Gramps itself.

It's quite a long time since I imported a GEDCOM from Ancestry myself,
but I do remember that they exported persons with IDs starting with a
letter P and not the more common I, and although there is no law that
says that person IDs should start with an I, the P may cause a problem,
because Gramps generates keys starting with P for places, and those IDs
are always generated by Gramps itself, since places don't have IDs in
GEDCOM at all.

For now, this is just a wild thought, but the GEDCOM standard says that
IDs must be unique to the file, and not just to the record type. And if
Gramps tries to maintain that standard, there may be a clash some times.

Can you tell what sort of Ancestry IDs are lost in the GEDCOM or SQLite
database exported by Gramps? Is there a pattern somewhere?

I think that Gramps will not show errors when new IDs are generated,
because generating new IDs in normal when you import a GEDCOM into a non
empty tree.

I bet that it should be simple for you to generate a list of lost IDs
and send it to my personal email, so that I can give them a glance.

regards,

Enno

>
> I regularly look at all of the diagnostic messages which flow from the
> import/load, and am familiar with the reasons that Gramps drops or
> otherwise notices incoming content that it does not like.  There are no
> diagnostic notices that might explain why some FAMS tag might have been
> found to be invalid and some substitution to have been made in order,
> for example, to preserve the necessary relational integrity as between
> Persons and Families.
>
> I am not stating that I think Gramps logic is incorrect, indeed, since
> the new circumstance only showed up after performing changes to the
> source gedcom at Ancestry, the most likely cause of the new and
> unexpected behavior, is on the incoming data side of things. Obviously
> on a continuing basis, additions and deletions of Persons and Families,
> causes the Ancestry database maintenance code to sometimes dynamically
> re-assign its own keys.  So, as between point-in-time #One and
> point-in-time #Two I could observe that the Ancestry.com-assigned ID for
> an individual would change. That is expected behavior and works fine as
> long as the relational integrity between tables/views remains correct.
> So, whatever my most recent additions and deletions have been, with
> whatever changes of the identifiers within Ancestry have been, that
> system remains consistent AND after the import, Gramps remain
> consistent.  The strangeness is only in respect of not understanding why
> Gramps has generated a handful of Family identifiers not supplied by the
> Ancestry source.
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> Gramps-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-users
> https://gramps-project.org


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Loading...