Filter performance?

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Filter performance?

paul womack
I am attempting to answer a simple question (it's a real question).

A friend on Facebook is also researching his family, and also
has some ancestors in a village in which I have ancestors, but
he didn't have any with my surname (so he thought we didn't have any
shared ancestors).

"obviously" I have many ancestors who don't share my surname...

So I thought I would generate a list of all the surnames
of my relations (not just ancestors) in that village (actually
pair of villages very close together).

So I started with a Place filter, called myPlace,
which was 4 filters in all; Places matching (id1), Places
enclosed by (id1) and ditto for village 2, 4 rules in all.

At least 1 rule must apply (an "OR")

(sidebar; will be only 2 rules in future

https://gramps-project.org/bugs/view.php?id=9427

).

This gives 40 Places in all (churches, houses, streets, farms, it all adds up),
and runs in about 1 second.

I then created (OK, reused) an Event Filter "@myPlace"
which is very simple;

Events of places matching the <place filter> myPlace.

This gives 790 Events, and runs in about 5 seconds.

I then move to Person, and finally build the filter I really want.

It only has two parts; Relationship Filter -> People related to Id
(myself)
Persons with Event filter matching @myPlace
All Rules Must Apply (an "AND").

I don't know how long this takes to run; I aborted it after 10 minutes.

Ah; I was going to time the two parts, but it appears "related to me"
is going infinite on its own.

So I guess this is a bug report, not a performance report.

The DB has 2345 Persons, 644 Families, 7833 Events, 1724 Places, 4207 Citations, 3240 Media.

    BugBear

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filter performance?

paul womack
paul womack wrote:

> I don't know how long this takes to run; I aborted it after 10 minutes.
>
> Ah; I was going to time the two parts, but it appears "related to me"
> is going infinite on its own.
>
> So I guess this is a bug report, not a performance report.
>
> The DB has 2345 Persons, 644 Families, 7833 Events, 1724 Places, 4207 Citations, 3240 Media.

And it just ran in 13 seconds.

The infinite loop is not 100% repeatable. :-(

  BugBear


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: Filter performance?

adrian.davey
In reply to this post by paul womack
I regularly use a similar set of filters in a similar fashion to find
people with at least one event relating to a [nested] place similarly
conceived.

In my case I am on 4.2.3 on AIO32.

We are not helped by the lack of sort mechanism in the list of custom
filters ... and the difficulty of reviewing the entire ... AND (... OR
... ) etc syntax in one view when editing nesting filters! I find it
depressingly easy for the wrong syntax to sneak in somewhere along the
chain! And sometimes I have to go away and have a glass of wine (or two)
before I can see where I have blundered!

My db is 33K people, and 86K events and 2K places [if the filter doesn't
call them, I can't see that the number of families, citations etc would
have any effect at all] and yes it does take a while [so the first thing
I do when finished is tag the lot so I can return to that set much more
quickly next time].

I have not timed it, but my guess is in my case it finishes in about 5
minutes. By my standards, your db is tiny, so it should be a lot
quicker, as now seems to be confirmed by your 13s run.

For some weird reason that I have not yet got to the bottom of,
sometimes it takes two successive runs before it behaves correctly -
this might explain your subsequent run. My current suspicion is that it
depends on one or other of the starting sort-orders in any one or more
of the places/events/people being filtered.

It is possible, though I hope unlikely, that the "@" character in your
filter name is an issue. For example, in quite different parts of gramps
I have discovered the hard way that if there is an ampersand [the "&"
character] anywhere in the name of a place it completely destroys the
formatting of any chart containing that string! I haven't yet worked out
where and why, but it is fatal enough for me to have reluctantly edited
all ampersands out of placenames, even though I much prefer it to the
"+" character.

cheers, Adrian

On 2017-03-16 20:31, paul womack wrote:

> I am attempting to answer a simple question (it's a real question).
>
> A friend on Facebook is also researching his family, and also
> has some ancestors in a village in which I have ancestors, but
> he didn't have any with my surname (so he thought we didn't have any
> shared ancestors).
>
> "obviously" I have many ancestors who don't share my surname...
>
> So I thought I would generate a list of all the surnames
> of my relations (not just ancestors) in that village (actually
> pair of villages very close together).
>
> So I started with a Place filter, called myPlace,
> which was 4 filters in all; Places matching (id1), Places
> enclosed by (id1) and ditto for village 2, 4 rules in all.
>
> At least 1 rule must apply (an "OR")
>
> (sidebar; will be only 2 rules in future
>
> https://gramps-project.org/bugs/view.php?id=9427
>
> ).
>
> This gives 40 Places in all (churches, houses, streets, farms, it all adds up),
> and runs in about 1 second.
>
> I then created (OK, reused) an Event Filter "@myPlace"
> which is very simple;
>
> Events of places matching the <place filter> myPlace.
>
> This gives 790 Events, and runs in about 5 seconds.
>
> I then move to Person, and finally build the filter I really want.
>
> It only has two parts; Relationship Filter -> People related to Id
> (myself)
> Persons with Event filter matching @myPlace
> All Rules Must Apply (an "AND").
>
> I don't know how long this takes to run; I aborted it after 10 minutes.
>
> Ah; I was going to time the two parts, but it appears "related to me"
> is going infinite on its own.
>
> So I guess this is a bug report, not a performance report.
>
> The DB has 2345 Persons, 644 Families, 7833 Events, 1724 Places, 4207 Citations, 3240 Media.
>
>      BugBear
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gramps-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-users
> https://gramps-project.org
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org