slowness of the Narrative web report in 5.0a3

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

slowness of the Narrative web report in 5.0a3

brian fitzgerald
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I
migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops
and immediately ran the narrative web report. It has been chugging along
for 22hr now and has not yet completed the individual pages though many
have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up
things a bit..dependencies.... ?


brian


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Nick Hall
On 01/01/18 15:50, brian wrote:
> I was wondering if there is something I could check for to speed up
> things a bit..dependencies.... ?

Which backend are you using - BSDDB or SQLite?

Nick.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Serge Noiraud-2
In reply to this post by brian fitzgerald
Le 01/01/2018 à 16:50, brian a écrit :

> Has anyone noticed how slow the report is compared to 4.2.6?
>
> just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.
>
> The same report in 4.2.6 finished in 3.5 minutes.
>
> I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?
>
>
> brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Serge Noiraud-2
Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
Thanks Nick. Got it!

Serge,
 I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I estimate about 30% of that time is spent on the 1500 'media pages'.

The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2 anymore (I know what I do).
What is you database ? When you start gramps, In the database chooser, what is the third column value?

In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes about 24hr to get through the first 2000 'individual pages'. Filter is set to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how to pre-load, tune or disable it. There are some new parameters in the new GUI which I am unfamiliar with.

On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]> wrote:
Le 01/01/2018 à 16:50, brian a écrit :
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?


brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge
Serge
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Serge Noiraud-2
Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
BSDDB
In your case, try to export this database, create a new one and import it
In this case, is it better ?

Serge


On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud <[hidden email]> wrote:
Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
Thanks Nick. Got it!

Serge,
 I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I estimate about 30% of that time is spent on the 1500 'media pages'.

The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2 anymore (I know what I do).
What is you database ? When you start gramps, In the database chooser, what is the third column value?

In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes about 24hr to get through the first 2000 'individual pages'. Filter is set to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how to pre-load, tune or disable it. There are some new parameters in the new GUI which I am unfamiliar with.

On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]> wrote:
Le 01/01/2018 à 16:50, brian a écrit :
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?


brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge
Serge



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

brian fitzgerald
Ok, I will try. 

On Jan 2, 2018 3:52 PM, "Serge Noiraud" <[hidden email]> wrote:
Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
BSDDB
In your case, try to export this database, create a new one and import it
In this case, is it better ?

Serge


On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud <[hidden email]> wrote:
Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
Thanks Nick. Got it!

Serge,
 I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I estimate about 30% of that time is spent on the 1500 'media pages'.

The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2 anymore (I know what I do).
What is you database ? When you start gramps, In the database chooser, what is the third column value?

In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes about 24hr to get through the first 2000 'individual pages'. Filter is set to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how to pre-load, tune or disable it. There are some new parameters in the new GUI which I am unfamiliar with.

On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]> wrote:
Le 01/01/2018 à 16:50, brian a écrit :
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?


brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge
Serge



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

prculley
I've noticed a few things about Narrative Web speed.

1) if you use filters it will slow down a lot.  On my system, if I enable the 'Privacy' filter, it takes nearly 3x the time as without it. The 'Living' filter is almost as bad.

2) If you have a lot of media, things slow down a lot.  NarWeb uses a thumbnail cache ("~/.gramps/thumb" for Linux, or "C:\Users\you\AppData\Roaming\gramps\thumb" for Windows) to speed up repetitive generations of the web site.  But the first time on a new machine or with a new tree (freshly imported from Gedcom, or manually generated), will be very slow while the thumbnails are created the first time.  The thumbnails are actually created during the person pages step, so that seems to take a long time.  But subsequent attempts to run NarWeb on the same tree will go much faster.  If you only use Gramps as a way to generate a narrative web, and import from Gedcom each time you will suffer the time each time (and your thumbnail cache will continue to get bigger with multiple copies of each thumbnail).

I'm not yet convinced that alpha3 is slower than 4.2.6, except for the new 'statistics' page, which takes a fair amount of time due to the way it interacts with the filter proxies.

Paul C.

On Tue, Jan 2, 2018 at 2:55 PM, brian fitzgerald <[hidden email]> wrote:
Ok, I will try. 

On Jan 2, 2018 3:52 PM, "Serge Noiraud" <[hidden email]> wrote:
Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
BSDDB
In your case, try to export this database, create a new one and import it
In this case, is it better ?

Serge


On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud <[hidden email]> wrote:
Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
Thanks Nick. Got it!

Serge,
 I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I estimate about 30% of that time is spent on the 1500 'media pages'.

The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2 anymore (I know what I do).
What is you database ? When you start gramps, In the database chooser, what is the third column value?

In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes about 24hr to get through the first 2000 'individual pages'. Filter is set to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how to pre-load, tune or disable it. There are some new parameters in the new GUI which I am unfamiliar with.

On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]> wrote:
Le 01/01/2018 à 16:50, brian a écrit :
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?


brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge
Serge



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

J. Allen Crider-2
This sounds consistent with what I've seen in 4.2.6 (I haven't tried 5.0
yet).  My database has over 55000 people and over 8000 media objects.
Generating a web site with no filtering about two weeks ago took less
than two hours on my system.  Excluding private records and only
including names for living persons increased the time to nearly seven
hours when I generated a new web site Monday evening.  The Dynamic Web
Report is much faster, but there are a few details I don't like as well,
so I haven't decided to switch yet.

Allen Crider

On 01/03/2018 09:43 AM, Paul Culley wrote:

> I've noticed a few things about Narrative Web speed.
>
> 1) if you use filters it will slow down a lot.  On my system, if I
> enable the 'Privacy' filter, it takes nearly 3x the time as without it.
> The 'Living' filter is almost as bad.
>
> 2) If you have a lot of media, things slow down a lot.  NarWeb uses a
> thumbnail cache ("~/.gramps/thumb" for Linux, or
> "C:\Users\you\AppData\Roaming\gramps\thumb" for Windows) to speed up
> repetitive generations of the web site.  But the first time on a new
> machine or with a new tree (freshly imported from Gedcom, or manually
> generated), will be very slow while the thumbnails are created the first
> time.  The thumbnails are actually created during the person pages step,
> so that seems to take a long time.  But subsequent attempts to run
> NarWeb on the same tree will go much faster.  If you only use Gramps as
> a way to generate a narrative web, and import from Gedcom each time you
> will suffer the time each time (and your thumbnail cache will continue
> to get bigger with multiple copies of each thumbnail).
>
> I'm not yet convinced that alpha3 is slower than 4.2.6, except for the
> new 'statistics' page, which takes a fair amount of time due to the way
> it interacts with the filter proxies.
>
> Paul C.
>
> On Tue, Jan 2, 2018 at 2:55 PM, brian fitzgerald <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Ok, I will try.
>
>     On Jan 2, 2018 3:52 PM, "Serge Noiraud" <[hidden email]
>     <mailto:[hidden email]>> wrote:
>
>         Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
>>         BSDDB
>         In your case, try to export this database, create a new one and
>         import it
>         In this case, is it better ?
>
>         Serge
>
>>
>>         On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud
>>         <[hidden email] <mailto:[hidden email]>> wrote:
>>
>>             Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
>>>             Thanks Nick. Got it!
>>>
>>>             Serge,
>>>              I have about 6000 people. Under 4.2.6 it completes in
>>>             226 seconds. I estimate about 30% of that time is spent
>>>             on the 1500 'media pages'.
>>>
>>             The 105 seconds for my 8430 people was with 5.0a3. I don't
>>             use 4.2 anymore (I know what I do).
>>             What is you database ? When you start gramps, In the
>>             database chooser, what is the third column value?
>>
>>>             In 5.0.0a3 with new import of the above data (from my
>>>             4.2.6) it takes about 24hr to get through the first 2000
>>>             'individual pages'. Filter is set to 'entire database'
>>>             .Might be 'cache-thrash'. Do you have any advice on how
>>>             to pre-load, tune or disable it. There are some new
>>>             parameters in the new GUI which I am unfamiliar with.
>>>
>>>             On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud
>>>             <[hidden email] <mailto:[hidden email]>> wrote:
>>>
>>>                 Le 01/01/2018 à 16:50, brian a écrit :
>>>
>>>                     Has anyone noticed how slow the report is
>>>                     compared to 4.2.6?
>>>
>>>                     just began some testing  since I had a few day
>>>                     over the Holidays and I migrated my 4.2.6 data to
>>>                     a clean version of 5.0a3. I checked for loops and
>>>                     immediately ran the narrative web report. It has
>>>                     been chugging along for 22hr now and has not yet
>>>                     completed the individual pages though many have
>>>                     been generated.
>>>
>>>                     The same report in 4.2.6 finished in 3.5 minutes.
>>>
>>>                     I was wondering if there is something I could
>>>                     check for to speed up things a
>>>                     bit..dependencies.... ?
>>>
>>>
>>>                     brian
>>>
>>>                 How many people do you have ?
>>>                 How many media do you have ?
>>>
>>>                 For me, If I create a navweb without media, It takes
>>>                 105 seconds for my 8430 people.
>>>                 If I add my 333 media, it takes 531 secondes (~1.8GB)
>>>                 My database is a BSDDB.
>>>
>>>                 Serge
>>>
>>             Serge
>>
>>
>
>
>     ------------------------------------------------------------------------------
>     Check out the vibrant tech community on one of the world's most
>     engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>     _______________________________________________
>     Gramps-users mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     https://lists.sourceforge.net/lists/listinfo/gramps-users
>     <https://lists.sourceforge.net/lists/listinfo/gramps-users>
>     https://gramps-project.org
>
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>
>
>
> _______________________________________________
> Gramps-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-users
> https://gramps-project.org
>

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Dave Scheipers
In reply to this post by prculley
I too have noticed that the generation of the media pages takes the
longest time to process. A complex filter of people can also take a
long time but can be mitigated by exporting the filter to a new
database that can then be used to generate the NarWeb without a
filter.

Not knowing exactly what is happening in the code, it seems that the
process to create the media pages has the NarWeb generating the
thumbnails regardless if those thumbnails have already been created
and stored in "C:\Users\you\AppData\Roaming\gramps\thumb".

I recently made two small sites for a nephew. One using NarWeb and
another using Dynamic Web. While I did not time the operations the
DynamWeb was much faster generating the same media files than NarWeb.

I run the Thumbnail Generator at least once a week. And if I have made
changes to the underlying media files and their folders, I'll wipe out
the existing "C:\Users\you\AppData\Roaming\gramps\thumb" folders and
do a complete new Thumbnail Generation. This takes over night to do
(13K objects and growing). The last time I did a more major NarWeb
report, I had to run it overnight with the media pages taking most of
that time.

I've been toying with the idea to open a bug on this issue. Now that
the topic has been raised others may have an insight into what is
actually occurring.

Dave

4.2.6 on Win10



On Wed, Jan 3, 2018 at 10:43 AM, Paul Culley <[hidden email]> wrote:

> I've noticed a few things about Narrative Web speed.
>
> 1) if you use filters it will slow down a lot.  On my system, if I enable
> the 'Privacy' filter, it takes nearly 3x the time as without it. The
> 'Living' filter is almost as bad.
>
> 2) If you have a lot of media, things slow down a lot.  NarWeb uses a
> thumbnail cache ("~/.gramps/thumb" for Linux, or
> "C:\Users\you\AppData\Roaming\gramps\thumb" for Windows) to speed up
> repetitive generations of the web site.  But the first time on a new machine
> or with a new tree (freshly imported from Gedcom, or manually generated),
> will be very slow while the thumbnails are created the first time.  The
> thumbnails are actually created during the person pages step, so that seems
> to take a long time.  But subsequent attempts to run NarWeb on the same tree
> will go much faster.  If you only use Gramps as a way to generate a
> narrative web, and import from Gedcom each time you will suffer the time
> each time (and your thumbnail cache will continue to get bigger with
> multiple copies of each thumbnail).
>
> I'm not yet convinced that alpha3 is slower than 4.2.6, except for the new
> 'statistics' page, which takes a fair amount of time due to the way it
> interacts with the filter proxies.
>
> Paul C.
>
> On Tue, Jan 2, 2018 at 2:55 PM, brian fitzgerald <[hidden email]> wrote:
>>
>> Ok, I will try.
>>
>> On Jan 2, 2018 3:52 PM, "Serge Noiraud" <[hidden email]> wrote:
>>>
>>> Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
>>>
>>> BSDDB
>>>
>>> In your case, try to export this database, create a new one and import it
>>> In this case, is it better ?
>>>
>>> Serge
>>>
>>>
>>> On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud <[hidden email]>
>>> wrote:
>>>>
>>>> Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
>>>>
>>>> Thanks Nick. Got it!
>>>>
>>>> Serge,
>>>>  I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I
>>>> estimate about 30% of that time is spent on the 1500 'media pages'.
>>>>
>>>> The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2
>>>> anymore (I know what I do).
>>>> What is you database ? When you start gramps, In the database chooser,
>>>> what is the third column value?
>>>>
>>>> In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes
>>>> about 24hr to get through the first 2000 'individual pages'. Filter is set
>>>> to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how
>>>> to pre-load, tune or disable it. There are some new parameters in the new
>>>> GUI which I am unfamiliar with.
>>>>
>>>> On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]>
>>>> wrote:
>>>>>
>>>>> Le 01/01/2018 à 16:50, brian a écrit :
>>>>>>
>>>>>> Has anyone noticed how slow the report is compared to 4.2.6?
>>>>>>
>>>>>> just began some testing  since I had a few day over the Holidays and I
>>>>>> migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and
>>>>>> immediately ran the narrative web report. It has been chugging along for
>>>>>> 22hr now and has not yet completed the individual pages though many have
>>>>>> been generated.
>>>>>>
>>>>>> The same report in 4.2.6 finished in 3.5 minutes.
>>>>>>
>>>>>> I was wondering if there is something I could check for to speed up
>>>>>> things a bit..dependencies.... ?
>>>>>>
>>>>>>
>>>>>> brian
>>>>>
>>>>> How many people do you have ?
>>>>> How many media do you have ?
>>>>>
>>>>> For me, If I create a navweb without media, It takes 105 seconds for my
>>>>> 8430 people.
>>>>> If I add my 333 media, it takes 531 secondes (~1.8GB)
>>>>> My database is a BSDDB.
>>>>>
>>>>> Serge
>>>>
>>>> Serge
>>>
>>>
>>>
>>
>>
>> ------------------------------------------------------------------------------
>> Check out the vibrant tech community on one of the world's most
>> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
>> _______________________________________________
>> Gramps-users mailing list
>> [hidden email]
>> https://lists.sourceforge.net/lists/listinfo/gramps-users
>> https://gramps-project.org
>
>
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Gramps-users mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-users
> https://gramps-project.org

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Serge Noiraud-2
In reply to this post by prculley
Le 03/01/2018 à 16:43, Paul Culley a écrit :
I've noticed a few things about Narrative Web speed.

1) if you use filters it will slow down a lot.  On my system, if I enable the 'Privacy' filter, it takes nearly 3x the time as without it. The 'Living' filter is almost as bad.

2) If you have a lot of media, things slow down a lot.  NarWeb uses a thumbnail cache ("~/.gramps/thumb" for Linux, or "C:\Users\you\AppData\Roaming\gramps\thumb" for Windows) to speed up repetitive generations of the web site.  But the first time on a new machine or with a new tree (freshly imported from Gedcom, or manually generated), will be very slow while the thumbnails are created the first time.  The thumbnails are actually created during the person pages step, so that seems to take a long time.  But subsequent attempts to run NarWeb on the same tree will go much faster.  If you only use Gramps as a way to generate a narrative web, and import from Gedcom each time you will suffer the time each time (and your thumbnail cache will continue to get bigger with multiple copies of each thumbnail).

I'm not yet convinced that alpha3 is slower than 4.2.6, except for the new 'statistics' page, which takes a fair amount of time due to the way it interacts with the filter proxies.
I am making a PR for multiple things which could be called "bugs" :
1 - I add an option for the statistics page.
2 - We have problems with notes : pages are too large
3 - Difficult to see some page on a mobile phone when you have notes.
4 - Resizing of images on a mobile
5 - maps are too bigs on a mobile


Paul C.

On Tue, Jan 2, 2018 at 2:55 PM, brian fitzgerald <[hidden email]> wrote:
Ok, I will try. 

On Jan 2, 2018 3:52 PM, "Serge Noiraud" <[hidden email]> wrote:
Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
BSDDB
In your case, try to export this database, create a new one and import it
In this case, is it better ?

Serge


On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud <[hidden email]> wrote:
Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
Thanks Nick. Got it!

Serge,
 I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I estimate about 30% of that time is spent on the 1500 'media pages'.

The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2 anymore (I know what I do).
What is you database ? When you start gramps, In the database chooser, what is the third column value?

In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes about 24hr to get through the first 2000 'individual pages'. Filter is set to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how to pre-load, tune or disable it. There are some new parameters in the new GUI which I am unfamiliar with.

On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]> wrote:
Le 01/01/2018 à 16:50, brian a écrit :
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?


brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge
Serge

Serge


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

paul womack
In reply to this post by Dave Scheipers
Dave Scheipers wrote:
> I too have noticed that the generation of the media pages takes the
> longest time to process. A complex filter of people can also take a
> long time but can be mitigated by exporting the filter to a new
> database that can then be used to generate the NarWeb without a
> filter.

Interesting - I think gramps has a habit of running filters
more often than you'd like.

I was recently deleting "most" events from a database, leaving only
a select few (around 10%).

The deletion went quite slowly; I noted that the event view
maintained the count of both the "selected" and "total" events
continuously during the process. I suspect this is done by running
the filter AFTER EACH DELETE!!

This implies that deleting (e.g.) people who are the result
of a complex filter (I have some filters that take 30 and more seconds to run)
would be "unfortunate"

  BugBear

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

Ron Johnson
In reply to this post by Dave Scheipers
On 01/04/2018 05:07 AM, paul womack wrote:
Dave Scheipers wrote:
I too have noticed that the generation of the media pages takes the
longest time to process. A complex filter of people can also take a
long time but can be mitigated by exporting the filter to a new
database that can then be used to generate the NarWeb without a
filter.

Interesting - I think gramps has a habit of running filters
more often than you'd like.

I was recently deleting "most" events from a database, leaving only
a select few (around 10%).

The deletion went quite slowly; I noted that the event view
maintained the count of both the "selected" and "total" events
continuously during the process. I suspect this is done by running
the filter AFTER EACH DELETE!!

I don't know about Windows, but on Linux the deletes will run a lot faster if you reduce the size of the window, so that less screen needs to be rewritten.  (I don't think that minimizing to the Panel helps.)

This implies that deleting (e.g.) people who are the result
of a complex filter (I have some filters that take 30 and more seconds to run)
would be "unfortunate"

-- 
World Peace Through Nuclear Pacification

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org
Reply | Threaded
Open this post in threaded view
|

Re: slowness of the Narrative web report in 5.0a3

brian fitzgerald
In reply to this post by Serge Noiraud-2

Confirmed that even with a SQLite database the individual pages take 24+ hrs to complete after a new install and a fresh import of known good archive. I pre-loaded the thumbnail 'cache' before begining the report and that took 2 minutes. However, I am happy to say that after the 24hr every person in the database has been relationship computed to every other one! That is Awesome! The narrative web report crashed in the 25th hour.

I am going to repeat the experiment with everything contained on the same media to exclude any networking related issues.


brian


On 01/03/2018 04:57 PM, Serge Noiraud wrote:
Le 03/01/2018 à 16:43, Paul Culley a écrit :
I've noticed a few things about Narrative Web speed.

1) if you use filters it will slow down a lot.  On my system, if I enable the 'Privacy' filter, it takes nearly 3x the time as without it. The 'Living' filter is almost as bad.

2) If you have a lot of media, things slow down a lot.  NarWeb uses a thumbnail cache ("~/.gramps/thumb" for Linux, or "C:\Users\you\AppData\Roaming\gramps\thumb" for Windows) to speed up repetitive generations of the web site.  But the first time on a new machine or with a new tree (freshly imported from Gedcom, or manually generated), will be very slow while the thumbnails are created the first time.  The thumbnails are actually created during the person pages step, so that seems to take a long time.  But subsequent attempts to run NarWeb on the same tree will go much faster.  If you only use Gramps as a way to generate a narrative web, and import from Gedcom each time you will suffer the time each time (and your thumbnail cache will continue to get bigger with multiple copies of each thumbnail).

I'm not yet convinced that alpha3 is slower than 4.2.6, except for the new 'statistics' page, which takes a fair amount of time due to the way it interacts with the filter proxies.
I am making a PR for multiple things which could be called "bugs" :
1 - I add an option for the statistics page.
2 - We have problems with notes : pages are too large
3 - Difficult to see some page on a mobile phone when you have notes.
4 - Resizing of images on a mobile
5 - maps are too bigs on a mobile


Paul C.

On Tue, Jan 2, 2018 at 2:55 PM, brian fitzgerald <[hidden email]> wrote:
Ok, I will try. 

On Jan 2, 2018 3:52 PM, "Serge Noiraud" <[hidden email]> wrote:
Le 02/01/2018 à 17:01, brian fitzgerald a écrit :
BSDDB
In your case, try to export this database, create a new one and import it
In this case, is it better ?

Serge


On Tue, Jan 2, 2018 at 10:55 AM, Serge Noiraud <[hidden email]> wrote:
Le 02/01/2018 à 16:10, brian fitzgerald a écrit :
Thanks Nick. Got it!

Serge,
 I have about 6000 people. Under 4.2.6 it completes in 226 seconds. I estimate about 30% of that time is spent on the 1500 'media pages'.

The 105 seconds for my 8430 people was with 5.0a3. I don't use 4.2 anymore (I know what I do).
What is you database ? When you start gramps, In the database chooser, what is the third column value?

In 5.0.0a3 with new import of the above data (from my 4.2.6) it takes about 24hr to get through the first 2000 'individual pages'. Filter is set to 'entire database' .Might be 'cache-thrash'. Do you have any advice on how to pre-load, tune or disable it. There are some new parameters in the new GUI which I am unfamiliar with.

On Tue, Jan 2, 2018 at 7:19 AM, Serge Noiraud <[hidden email]> wrote:
Le 01/01/2018 à 16:50, brian a écrit :
Has anyone noticed how slow the report is compared to 4.2.6?

just began some testing  since I had a few day over the Holidays and I migrated my 4.2.6 data to a clean version of 5.0a3. I checked for loops and immediately ran the narrative web report. It has been chugging along for 22hr now and has not yet completed the individual pages though many have been generated.

The same report in 4.2.6 finished in 3.5 minutes.

I was wondering if there is something I could check for to speed up things a bit..dependencies.... ?


brian
How many people do you have ?
How many media do you have ?

For me, If I create a navweb without media, It takes 105 seconds for my 8430 people.
If I add my 333 media, it takes 531 secondes (~1.8GB)
My database is a BSDDB.

Serge
Serge

Serge



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Gramps-users mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-users
https://gramps-project.org