GraphViz plugin changes/improvements

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

Attached is a patch to the Gramps GraphViz plugin that contains following
changes:

Bugfixes:
- Quote node and edge identifiers
  -> fixes problem with Gramps IDs having non-alphanumeric characters
- Make latin-1 conversion option explicit to user, not depending on font
  -> Fix for the text encoding problem with fonts & output formats

Feature requests:
- Women have rounded boxes, except in filled mode
  (current GraphViz doesn't support boxes that are both rounded and filled)

Re-factoring:
- Split large functions to smaller ones to make code more readable
- Renamed few methods and variables to what they actually do
- Collect the report text to a buffer first so that it can be converted to
  latin-1 at one go. This could allow later on to modify the buffer in place
  without regenerating the report (the output is normally some tens of KBs
  so I don't think it's a problem, but it's easy to revert)
- Move common style stuff to GraphViz header and remove from .dot
  comments stuff that's obvious from first .dot file settings
  -> Smaller GraphViz file (helps with above)

Please test.


I will do the other GraphViz options once I know whether the GUI/widget
helpers will be used in Gramps 2.2.

The stuff required for sorting the nodes is bit more complicated than I
thought because in GraphViz plugin you have actually two graphs to deal
with, persons and families.  Only at output phase they are joined and that's
done by GraphViz itself, not by GraphViz plugin, so it will need more
thought.  The refactoring I did should make it a bit easier though.


Btw. I was wondering, would it be possible to move the page settings to
the paper settings tab?  The paper settings seem to be created by Gramps.


        - Eero

GraphViz.py.diff (21K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Alex Roitman
Eero,

On Mon, 2006-01-23 at 23:31 +0200, Eero Tamminen wrote:

> Attached is a patch to the Gramps GraphViz plugin that contains following
> changes:
>
> Bugfixes:
> - Quote node and edge identifiers
>   -> fixes problem with Gramps IDs having non-alphanumeric characters
> - Make latin-1 conversion option explicit to user, not depending on font
>   -> Fix for the text encoding problem with fonts & output formats
>
> Feature requests:
> - Women have rounded boxes, except in filled mode
>   (current GraphViz doesn't support boxes that are both rounded and filled)
>
> Re-factoring:
> - Split large functions to smaller ones to make code more readable
> - Renamed few methods and variables to what they actually do
> - Collect the report text to a buffer first so that it can be converted to
>   latin-1 at one go. This could allow later on to modify the buffer in place
>   without regenerating the report (the output is normally some tens of KBs
>   so I don't think it's a problem, but it's easy to revert)
> - Move common style stuff to GraphViz header and remove from .dot
>   comments stuff that's obvious from first .dot file settings
>   -> Smaller GraphViz file (helps with above)
>
> Please test.
Maybe a better idea would be to email the whole file,
so that people can place it into ~/.gramps/plugins and test
without having the source built/installed.

I'll let you do that, in case there are further modifications
you want to share.

Alex

--
Alexander Roitman   http://www.gramps-project.org

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
On Thursday 26 January 2006 20:29, Alex Roitman wrote:

> > Attached is a patch to the Gramps GraphViz plugin that contains
> > following changes:
> >
> > Bugfixes:
> > - Quote node and edge identifiers
> >   -> fixes problem with Gramps IDs having non-alphanumeric characters
> > - Make latin-1 conversion option explicit to user, not depending on
> > font -> Fix for the text encoding problem with fonts & output formats
> >
> > Feature requests:
> > - Women have rounded boxes, except in filled mode
> >   (current GraphViz doesn't support boxes that are both rounded and
> > filled)
> >
> > Re-factoring:
> > - Split large functions to smaller ones to make code more readable
> > - Renamed few methods and variables to what they actually do
> > - Collect the report text to a buffer first so that it can be converted
> > to latin-1 at one go. This could allow later on to modify the buffer in
> > place without regenerating the report (the output is normally some tens
> > of KBs so I don't think it's a problem, but it's easy to revert)
> > - Move common style stuff to GraphViz header and remove from .dot
> >   comments stuff that's obvious from first .dot file settings
> >   -> Smaller GraphViz file (helps with above)
> >
> > Please test.
>
> Maybe a better idea would be to email the whole file,
> so that people can place it into ~/.gramps/plugins and test
> without having the source built/installed.
Ok, the whole file is attached.


> I'll let you do that, in case there are further modifications
> you want to share.

Well, first I'd like to know whether Doug's widget stuff will get
adopted? :-)


        - Eero

GraphViz.py (40K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Alex Roitman
On Fri, 2006-01-27 at 22:01 +0200, Eero Tamminen wrote:
> > I'll let you do that, in case there are further modifications
> > you want to share.
>
> Well, first I'd like to know whether Doug's widget stuff will get
> adopted? :-)

Neither Don nor myself have anything against Doug's widgets.
However, it's not a trivial piece of code, so the whole issue
is whether Doug feels committed to maintaining this code.

Doug, if you feel like maintaining this in the future then
let's add this. At this point, it's completely your call.

Alex

--
Alexander Roitman   http://www.gramps-project.org

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Doug Blank
> On Fri, 2006-01-27 at 22:01 +0200, Eero Tamminen wrote:
>> > I'll let you do that, in case there are further modifications
>> > you want to share.
>>
>> Well, first I'd like to know whether Doug's widget stuff will get
>> adopted? :-)
>
> Neither Don nor myself have anything against Doug's widgets.
> However, it's not a trivial piece of code, so the whole issue
> is whether Doug feels committed to maintaining this code.
>
> Doug, if you feel like maintaining this in the future then
> let's add this. At this point, it's completely your call.

If we have these widgets, it will mean that we shouldn't have to change a
majority of the reports if there is a change in gtk, or something in the
reports interface. Right now, if there is such a change, we will have to
go through and change each report. I actually don't know how often we have
to fiddle with that interface. Also, it will make reports much easier to
write.

I don't have a good feel for what all of the widgets will look like, yet.

I'll do this: I'll convert the graphviz report next, but without making
the more general changes. Then we'll have a better idea of what it takes
before I commit to effectively maintaining all widgets for all reports.

How's that sound?

-Doug

> Alex
>
> --
> Alexander Roitman   http://www.gramps-project.org
>




-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Alex Roitman
Doug,

On Fri, 2006-01-27 at 16:14 -0500, [hidden email] wrote:

> > Doug, if you feel like maintaining this in the future then
> > let's add this. At this point, it's completely your call.
>
> If we have these widgets, it will mean that we shouldn't have to change a
> majority of the reports if there is a change in gtk, or something in the
> reports interface. Right now, if there is such a change, we will have to
> go through and change each report. I actually don't know how often we have
> to fiddle with that interface. Also, it will make reports much easier to
> write.
>
> I don't have a good feel for what all of the widgets will look like, yet.
>
> I'll do this: I'll convert the graphviz report next, but without making
> the more general changes. Then we'll have a better idea of what it takes
> before I commit to effectively maintaining all widgets for all reports.
>
> How's that sound?
Sounds great to me.
Alex

--
Alexander Roitman   http://www.gramps-project.org

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
In reply to this post by Doug Blank
Hi,

On Friday 27 January 2006 23:14, [hidden email] wrote:

> >> > I'll let you do that, in case there are further modifications
> >> > you want to share.
> >>
> >> Well, first I'd like to know whether Doug's widget stuff will get
> >> adopted? :-)
> >
> > Neither Don nor myself have anything against Doug's widgets.
> > However, it's not a trivial piece of code, so the whole issue
> > is whether Doug feels committed to maintaining this code.
> >
> > Doug, if you feel like maintaining this in the future then
> > let's add this. At this point, it's completely your call.
>
> If we have these widgets, it will mean that we shouldn't have to change a
> majority of the reports if there is a change in gtk, or something in the
> reports interface. Right now, if there is such a change, we will have to
> go through and change each report. I actually don't know how often we
> have to fiddle with that interface. Also, it will make reports much
> easier to write.
>
> I don't have a good feel for what all of the widgets will look like, yet.
>
> I'll do this: I'll convert the graphviz report next, but without making
> the more general changes. Then we'll have a better idea of what it takes
> before I commit to effectively maintaining all widgets for all reports.
>
> How's that sound?

Excellent!

How you've thought to do the tab addition?

Currently (I think) the first tab comes from the Gramps, but I would like
the last tab contents (onto how many pages the graph should be output)
also to be on the first tab as IMHO it would be most logical to have all
"paper" options in the same place.


        - Eero


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

On Saturday 28 January 2006 01:15, Eero Tamminen wrote:
> > If we have these widgets, it will mean that we shouldn't have to change
> > a majority of the reports if there is a change in gtk, or something in
> > the reports interface.

And I guess this is going to help with the gramps-tk too. :-)

Statistics Chart report has some things that might be hard to do with
automatic widgets, like having checkbuttons in two columns.  Maybe it's
better if for those kind of reports the GUI is still done with Gtk directly,
or that there's at least a possibility to do part (one tab?) of GUI with Gtk
directly?


> > Right now, if there is such a change, we will
> > have to go through and change each report. I actually don't know how
> > often we have to fiddle with that interface. Also, it will make reports
> > much easier to write.
> >
> > I don't have a good feel for what all of the widgets will look like,
> > yet.
> >
> > I'll do this: I'll convert the graphviz report next, but without making
> > the more general changes. Then we'll have a better idea of what it
> > takes before I commit to effectively maintaining all widgets for all
> > reports.
> >
> > How's that sound?
>
> Excellent!
>
> How you've thought to do the tab addition?
>
> Currently (I think) the first tab comes from the Gramps, but I would like
> the last tab contents (onto how many pages the graph should be output)
> also to be on the first tab as IMHO it would be most logical to have all
> "paper" options in the same place.

The command line report stuff can be also a bit problematic, how to
integrate calling of an external program to the widget automation.

Btw. In current GraphViz code, "latin" is an explicit option from the
command line, but deduced from the other options in the GUI mode. You can
ignore the latter in your refactoring, in my version user sets that option
explicitly also in the GUI as that has turned out better (whether
utf8->latin1 conversion is needed depends both from selected font *and*
output format and it might be possible that there are latin1 and unicode
fonts under the same name i.e. font name might not suffice, so it's better
that user sets it himself).


        - Eero


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Alex Roitman
On Sun, 2006-01-29 at 11:31 +0200, Eero Tamminen wrote:

> On Saturday 28 January 2006 01:15, Eero Tamminen wrote:
> > > If we have these widgets, it will mean that we shouldn't have to change
> > > a majority of the reports if there is a change in gtk, or something in
> > > the reports interface.
>
> And I guess this is going to help with the gramps-tk too. :-)
>
> Statistics Chart report has some things that might be hard to do with
> automatic widgets, like having checkbuttons in two columns.  Maybe it's
> better if for those kind of reports the GUI is still done with Gtk directly,
> or that there's at least a possibility to do part (one tab?) of GUI with Gtk
> directly?
As far as I understood the widget code, it does not have to replace the
current framework, but rather builds on it. So if we have a (maintained)
GrampsWidgets module then the reports can choose whether or not to use
it. It definitely would simplify creation and maintenance of simple
reports.

> The command line report stuff can be also a bit problematic, how to
> integrate calling of an external program to the widget automation.
>
> Btw. In current GraphViz code, "latin" is an explicit option from the
> command line, but deduced from the other options in the GUI mode. You can
> ignore the latter in your refactoring, in my version user sets that option
> explicitly also in the GUI as that has turned out better (whether
> utf8->latin1 conversion is needed depends both from selected font *and*
> output format and it might be possible that there are latin1 and unicode
> fonts under the same name i.e. font name might not suffice, so it's better
> that user sets it himself).
I agree, and I thought your patch took care of that by introducing
"latin" as a GUI option, right?

Alex

--
Alexander Roitman   http://www.gramps-project.org

signature.asc (196 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

On Sunday 29 January 2006 20:57, Alex Roitman wrote:

> > Btw. In current GraphViz code, "latin" is an explicit option from the
> > command line, but deduced from the other options in the GUI mode. You
> > can ignore the latter in your refactoring, in my version user sets that
> > option explicitly also in the GUI as that has turned out better
> > (whether utf8->latin1 conversion is needed depends both from selected
> > font *and* output format and it might be possible that there are latin1
> > and unicode fonts under the same name i.e. font name might not suffice,
> > so it's better that user sets it himself).
>
> I agree, and I thought your patch took care of that by introducing
> "latin" as a GUI option, right?

Yes.


        - Eero


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

Attached is a slightly updated version of the refactored GraphViz plugin.
When I moved the attributes from individual nodes & edges to the GraphViz
header, I hadn't had put the attributes correctly inside node & edge
objects, now the arrow styles and font selection should again work.

I also set the ratio to "compress" instead of "fill".  It looks much better
as graph, but for some reason when I view the result in GGV, the page
is not of the specified size.  If I set ratio to "auto", it can fill
several pages. :-/   Do you have any advise on this?


        - Eero

GraphViz.py.gz (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

On Monday 30 January 2006 00:25, Eero Tamminen wrote:
> Attached is a slightly updated version of the refactored GraphViz plugin.
> When I moved the attributes from individual nodes & edges to the GraphViz
> header, I hadn't had put the attributes correctly inside node & edge
> objects, now the arrow styles and font selection should again work.
>
> I also set the ratio to "compress" instead of "fill".  It looks much
> better as graph, but for some reason when I view the result in GGV, the
> page is not of the specified size.  If I set ratio to "auto", it can fill
> several pages. :-/   Do you have any advise on this?

Attached is again a new version of the GraphViz plugin, just gunzip
it and put into your ~/.gramps/plugins/.

New user options:
- User can set the GraphViz aspect ratio setting.
- The weight option (either for children or parents) asked earlier. However,
  at least with my database weights doesn't seem to have any noticable
  effect.

New default settings:
- The plugin will use always "mclimit=2.0" setting, I noticed that this
  prevents crossed lines in my database, which makes it look better.
- Minimum node separation is 0.25. With the now default compress
  aspect ratio, the nodes would otherwise IMHO be a bit too close together.


        - Eero

GraphViz.py.gz (12K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

wb-3
That Graphviz plugin seems to work just fine.

However, did you know that if a person has a nickname,
specified like 'Joe "Big Joe" User', the graphviz output
fails?

I always have to go through the labels with a little program
to escape all of the quotes with backslashes.  So, that it
becomes: 'Joe \"Big Joe\" User'.  It's a small thing, but I
thought I should point it out to you.

                -- Wayne Bergeron


On Monday February 6 2006 15:59, Eero Tamminen wrote:

> Hi,
>
> On Monday 30 January 2006 00:25, Eero Tamminen wrote:
> > Attached is a slightly updated version of the
> > refactored GraphViz plugin. When I moved the attributes
> > from individual nodes & edges to the GraphViz header, I
> > hadn't had put the attributes correctly inside node &
> > edge objects, now the arrow styles and font selection
> > should again work.
> >
> > I also set the ratio to "compress" instead of "fill".
> > It looks much better as graph, but for some reason when
> > I view the result in GGV, the page is not of the
> > specified size.  If I set ratio to "auto", it can fill
> > several pages. :-/   Do you have any advise on this?
>
> Attached is again a new version of the GraphViz plugin,
> just gunzip it and put into your ~/.gramps/plugins/.
>
> New user options:
> - User can set the GraphViz aspect ratio setting.
> - The weight option (either for children or parents)
> asked earlier. However, at least with my database weights
> doesn't seem to have any noticable effect.
>
> New default settings:
> - The plugin will use always "mclimit=2.0" setting, I
> noticed that this prevents crossed lines in my database,
> which makes it look better. - Minimum node separation is
> 0.25. With the now default compress aspect ratio, the
> nodes would otherwise IMHO be a bit too close together.
>
>
> - Eero

--
        If we do not succeed, then we run the risk of failure.
       
                -- Vice President Dan Quayle, to the Phoenix
                         Republican Forum, 3/23/90 (reported in Esquire,
                         8/92) Also reported by Reuters, 5/2/90



-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

On Thursday 09 February 2006 21:08, wb wrote:

> That Graphviz plugin seems to work just fine.
>
> However, did you know that if a person has a nickname,
> specified like 'Joe "Big Joe" User', the graphviz output
> fails?
>
> I always have to go through the labels with a little program
> to escape all of the quotes with backslashes.  So, that it
> becomes: 'Joe \"Big Joe\" User'.  It's a small thing, but I
> thought I should point it out to you.
Fixes and new features in the attached version:
- Double quotes in names are escaped.  Should anything else be escaped
  in addition to names?  (dates?)
- User can add a (e.g. copyright) note to the graph and set its location
  (newlines and double quotes are automatically escaped)
- User can give the paging direction
- If converting labels to latin1, set the GraphViz charset attribute
  accordingly

Did you have any comments on the mclimit or using ratio=compress
as default or giving 0.25 as node separation?

Also, does the weights have an effect for your grouping of children/parents?
To me it didn't seem to do anything, and I'll remove the option if it's not
useful...


        - Eero

> -- Wayne Bergeron
>
> On Monday February 6 2006 15:59, Eero Tamminen wrote:
> > Hi,
> >
> > On Monday 30 January 2006 00:25, Eero Tamminen wrote:
> > > Attached is a slightly updated version of the
> > > refactored GraphViz plugin. When I moved the attributes
> > > from individual nodes & edges to the GraphViz header, I
> > > hadn't had put the attributes correctly inside node &
> > > edge objects, now the arrow styles and font selection
> > > should again work.
> > >
> > > I also set the ratio to "compress" instead of "fill".
> > > It looks much better as graph, but for some reason when
> > > I view the result in GGV, the page is not of the
> > > specified size.  If I set ratio to "auto", it can fill
> > > several pages. :-/   Do you have any advise on this?
> >
> > Attached is again a new version of the GraphViz plugin,
> > just gunzip it and put into your ~/.gramps/plugins/.
> >
> > New user options:
> > - User can set the GraphViz aspect ratio setting.
> > - The weight option (either for children or parents)
> > asked earlier. However, at least with my database weights
> > doesn't seem to have any noticable effect.
> >
> > New default settings:
> > - The plugin will use always "mclimit=2.0" setting, I
> > noticed that this prevents crossed lines in my database,
> > which makes it look better. - Minimum node separation is
> > 0.25. With the now default compress aspect ratio, the
> > nodes would otherwise IMHO be a bit too close together.
> >
> >
> > - Eero

GraphViz.py.gz (13K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Julio Sánchez-2
2006/2/11, Eero Tamminen <[hidden email]>:
> Fixes and new features in the attached version:
> - Double quotes in names are escaped.  Should anything else be escaped
>   in addition to names?  (dates?)

Not sure, but on labels you need to escape as well: <, >, |, {, }

The first three maybe only break on records.  The last two break
everywere (I use them to mark implied surnames).

On IDs you had to escape something, '-' I think.  IDs from
familysearch, Vital Record Index, etc. contain that character that,
anyway, is legal in a GEDCOM ID.  So, if GEDCOM IDs are used to build
GraphViz ID's any characted possible in the former and not the latter
has to be escaped.

All these characters seem to break graphviz.

Also, it is important that no Latin1 character with its high bit set
ever appears right before \n or the end of string (that it will see as
a terminating NUL).  There is a bug in some GraphViz versions that
will gobble blindly the following character.  In the case of the end
of string, it will scan beyond the end of string producing random
results or crashes.

> - If converting labels to latin1, set the GraphViz charset attribute
>   accordingly

I suppose this is what makes sense now, but really graphviz should
take care of this if needed.

> Did you have any comments on the mclimit or using ratio=compress
> as default or giving 0.25 as node separation?
>
> Also, does the weights have an effect for your grouping of children/parents?
> To me it didn't seem to do anything, and I'll remove the option if it's not
> useful...

I did not find that option, but I am still experimenting with it.
Anyway, I don't care much about this kind of graph, I only produce
graphs with spouses stacked together as records, so my feedback will
not be very complete. The first impression is that large graphs are
still hard to make sense of, even with the simpler layout requirements
of this kind of graphs.  I think edge coloring would help here as
well.

A few other things.

The graph orientation option is a misnomer.  Always has been, at least
since I first met GRAMPS.  It lets you set LR or RL rankdir and calls
them vertical and horizontal.  This is patently false, both are
similar and only differ in placing ancestors on the right or the left.
 There is another rankdir value, BT, that does something of that.  But
it would make sense only to build extra wide graphs and probably would
require help in stacking name pieces vertically so that boxes are as
narrow as possible.  For Spanish people, with long christening names
and multiple surnames, this is critical.

Also, the option to reduce dates to just years is not very helpful if
you put estimated ranges in birth or death dates since they will show
in the graph as absolute truths.  It does not matter that the help tip
explains this, the consumer of the graph does not get to see the
disclaimer.

I suggest using something in the line of the function implemented by
the attached patch to NameDisplay.py.

Also, having christening/burial events stand as estimated dates for
birth/death is very important.  The GraphViz.py version I sent had it
and I will always need it, even if it is only in my private version.

Regards,

Julio

DateDisplay-compact.patch (2K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

Attached GraphViz plugin has the following improvements:
- Fix for special characters (<,>,|,{,}) in female names
- Use christening/burial dates if birth/death dates are missing
- Rankdir options correspond now to TB and LR (earlier LR and RL)

More details below.


> > Fixes and new features in the attached version:
> > - Double quotes in names are escaped.  Should anything else be escaped
> >   in addition to names?  (dates?)
>
> Not sure, but on labels you need to escape as well: <, >, |, {, }

I didn't have any problem in using these in labels (just tested it).


> The first three maybe only break on records.

So it seems, wierd.

There's no problem if you use:
        shape=box, style=rounded
though, and this seems to look the same as Mrecord.


> The last two break everywere (I use them to mark implied surnames).
>
> On IDs you had to escape something, '-' I think.  IDs from
> familysearch, Vital Record Index, etc. contain that character that,
> anyway, is legal in a GEDCOM ID.  So, if GEDCOM IDs are used to build
> GraphViz ID's any characted possible in the former and not the latter
> has to be escaped.

I've fixed this two versions ago.  It was enough just to put the IDs into
quotes.

> All these characters seem to break graphviz.
>
> Also, it is important that no Latin1 character with its high bit set
> ever appears right before \n or the end of string (that it will see as
> a terminating NUL).  There is a bug in some GraphViz versions that
> will gobble blindly the following character.  In the case of the end
> of string, it will scan beyond the end of string producing random
> results or crashes.

I'm hoping the charset setting will fix this.  GraphViz default is UTF-8 and
I think the problem is that it doesn't validate whether the string is valid
UTF-8.


> > - If converting labels to latin1, set the GraphViz charset attribute
> >   accordingly
>
> I suppose this is what makes sense now, but really graphviz should
> take care of this if needed.

AFAIK it cannot.    Whether latin1 is needed depends completely on the font,
and the font is (in PostScript output format) something that your PostScript
interpreter handles, not Gramps or GraphViz.  Whereas with e.g. SVG output
the text should be always UTF-8.

It's better that user can fix the problem than that we get it wrong and
user cannot fix it.  Anyway, user probably will normally use the same output
format and font most of the time, so he doesn't need to change the setting
once he's gotten it right.


> > Did you have any comments on the mclimit or using ratio=compress
> > as default or giving 0.25 as node separation?
> >
> > Also, does the weights have an effect for your grouping of
> > children/parents? To me it didn't seem to do anything, and I'll remove
> > the option if it's not useful...
>
> I did not find that option, but I am still experimenting with it.

It's under "Layout" as "what to group together". Although it sets the
weights into edges, that doesn't seem to have any effect that I could notice
with the example database.


> Anyway, I don't care much about this kind of graph, I only produce
> graphs with spouses stacked together as records, so my feedback will
> not be very complete. The first impression is that large graphs are
> still hard to make sense of, even with the simpler layout requirements
> of this kind of graphs.  I think edge coloring would help here as
> well.

Could you explain what you mean by "edge coloring"?


> A few other things.
>
> The graph orientation option is a misnomer.  Always has been, at least
> since I first met GRAMPS.  It lets you set LR or RL rankdir and calls
> them vertical and horizontal.  This is patently false, both are
> similar and only differ in placing ancestors on the right or the left.

I've always wondered about that, but for me RL always produces vertical
graph.  (dot v1.11 bug?)

I changed the rankdir to use BT and LR which work correctly for me.
BT and TB seemed to produce the same results.


>  There is another rankdir value, BT, that does something of that.  But
> it would make sense only to build extra wide graphs and probably would
> require help in stacking name pieces vertically so that boxes are as
> narrow as possible.  For Spanish people, with long christening names
> and multiple surnames, this is critical.
>
> Also, the option to reduce dates to just years is not very helpful if
> you put estimated ranges in birth or death dates since they will show
> in the graph as absolute truths.  It does not matter that the help tip
> explains this, the consumer of the graph does not get to see the
> disclaimer.
>
> I suggest using something in the line of the function implemented by
> the attached patch to NameDisplay.py.
Could you send a similar patch against the attached version of
the GraphViz plugin?    The dates are now fetched in the get_date_strings()
method.


> Also, having christening/burial events stand as estimated dates for
> birth/death is very important.  The GraphViz.py version I sent had it
> and I will always need it, even if it is only in my private version.

I added this, but I hadn't data with which I could have tested it.
Could you test it (with the attached version)?



        - Eero

GraphViz.py.gz (14K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Julio Sánchez-2
2006/2/11, Eero Tamminen <[hidden email]>:

> > Not sure, but on labels you need to escape as well: <, >, |, {, }
>
> I didn't have any problem in using these in labels (just tested it).

Only failed in records, you got rid of them, the problem is gone.  I
still will be needing escaping them for records.

All those symbols have special meaning on record labels.

> There's no problem if you use:
>         shape=box, style=rounded
> though, and this seems to look the same as Mrecord.
>
>
> > The last two break everywere (I use them to mark implied surnames).

I was wrong on this, though.  They only failed in records.

> > On IDs you had to escape something, '-' I think.  IDs from
> > familysearch, Vital Record Index, etc. contain that character that,
> > anyway, is legal in a GEDCOM ID.  So, if GEDCOM IDs are used to build
> > GraphViz ID's any characted possible in the former and not the latter
> > has to be escaped.
>
> I've fixed this two versions ago.  It was enough just to put the IDs into
> quotes.

Great, did not think of that.

> > Also, it is important that no Latin1 character with its high bit set
> > ever appears right before \n or the end of string (that it will see as
> > a terminating NUL).  There is a bug in some GraphViz versions that
> > will gobble blindly the following character.  In the case of the end
> > of string, it will scan beyond the end of string producing random
> > results or crashes.
>
> I'm hoping the charset setting will fix this.  GraphViz default is UTF-8 and
> I think the problem is that it doesn't validate whether the string is valid
> UTF-8.

Old versions were more or less character-set blind, but tried to do
some supposedly smart thing with asian languages that was fatal for
Latin1.  Or something like that.  Don't remember the particulars, I
reported the bug to them, fixed my copy and never looked back.

> > I suppose this is what makes sense now, but really graphviz should
> > take care of this if needed.
>
> AFAIK it cannot.    Whether latin1 is needed depends completely on the font,
> and the font is (in PostScript output format) something that your PostScript
> interpreter handles, not Gramps or GraphViz.  Whereas with e.g. SVG output
> the text should be always UTF-8.

Precisely.  The GRAMPS user is required to know in advance what value
will be used for the -T option to dot in the future.  This nearly
costed me dearly last summer.  Fortunately I had with me everything
needed to rebuild the .dot file.  But I had not planned on having to
do this.

> It's better that user can fix the problem than that we get it wrong and
> user cannot fix it.  Anyway, user probably will normally use the same output
> format and font most of the time, so he doesn't need to change the setting
> once he's gotten it right.

When you say user, you mean "GRAMPS user", right?  But what about
"GraphViz user"?

They don't have to be the same, do they?

> It's under "Layout" as "what to group together". Although it sets the
> weights into edges, that doesn't seem to have any effect that I could notice
> with the example database.

I'll check it.  It worked for me in the past, only did not like the
effect back then.

> Could you explain what you mean by "edge coloring"?

Traditional hand-drawn trees use different leaf or branch styles for
the different lineages, this way you have visual clues that let you
move in the tree.  One such tree with 400 individuals hangs on a wall
in my living room.

Edge coloring is an approximation to this using GraphViz.  Works by
first determining the lineage of every individual.  You can use any
criteria for building lineages but the most useful by far is the
agnatic lineage, that is made up of all individuals descending by
direct male lines from some person.  In many cases, western people in
the same agnatic lineage share the same surname, but deviations are
common.

Then you assign a color to the most common lineages and make the edges
joining them have that color.

Another helpful hint is to make bold all edges from some distinguished
person in the graph (you, one of your children, etc).  This way you
can navigate easily from ancestors to descendants since you see easily
what branch will lead you down and will not end in a cul-de-sac.

I have code for all this (actually, it was included in the version of
GraphViz.py I sent, but only for records).  I think I sent a sample
graph with this just a few weeks ago.

> I've always wondered about that, but for me RL always produces vertical
> graph.  (dot v1.11 bug?)

I checked this morning with version 2.6, but I think it has always
been the same.  One orientation has descendants on the left and
ancestors on the right, the other flips them.  To produce vertical
progression, BT is needed.

> I changed the rankdir to use BT and LR which work correctly for me.
> BT and TB seemed to produce the same results.

TB is not documented, I think, but it would not be the first
undocumented option in GraphViz.

> > I suggest using something in the line of the function implemented by
> > the attached patch to NameDisplay.py.
>
> Could you send a similar patch against the attached version of
> the GraphViz plugin?    The dates are now fetched in the get_date_strings()
> method.

The patch was not a GraphViz patch, but a NameDisplay patch, so that
it can be refined for localization if needed.

GraphViz.py only needs to call the new function if years-only is requested.

> > Also, having christening/burial events stand as estimated dates for
> > birth/death is very important.  The GraphViz.py version I sent had it
> > and I will always need it, even if it is only in my private version.
>
> I added this, but I hadn't data with which I could have tested it.
> Could you test it (with the attached version)?

I will.

Julio


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

On Sunday 12 February 2006 00:52, Julio Sanchez wrote:
...

> > > I suppose this is what makes sense now, but really graphviz should
> > > take care of this if needed.
> >
> > AFAIK it cannot.    Whether latin1 is needed depends completely on the
> > font, and the font is (in PostScript output format) something that your
> > PostScript interpreter handles, not Gramps or GraphViz.  Whereas with
> > e.g. SVG output the text should be always UTF-8.
>
> Precisely.  The GRAMPS user is required to know in advance what value
> will be used for the -T option to dot in the future.  This nearly
> costed me dearly last summer.  Fortunately I had with me everything
> needed to rebuild the .dot file.  But I had not planned on having to
> do this.
>
> > It's better that user can fix the problem than that we get it wrong and
> > user cannot fix it.  Anyway, user probably will normally use the same
> > output format and font most of the time, so he doesn't need to change
> > the setting once he's gotten it right.
>
> When you say user, you mean "GRAMPS user", right?  But what about
> "GraphViz user"?
>
> They don't have to be the same, do they?

No, but if you're already using command line, doing:
        iconv -f iso-8859-1 -t utf-8 old.dot > new.dot
        iconv -t utf-8 -f iso-8859-1 old.dot > new.dot

Should be easy. :-)  Iconv is even mentioned in the GraphViz docs.

I added note about this to the comment in the generated .dot file.
Is there anything else that should be documented in the .dot file
header?


> > It's under "Layout" as "what to group together". Although it sets the
> > weights into edges, that doesn't seem to have any effect that I could
> > notice with the example database.
>
> I'll check it.  It worked for me in the past, only did not like the
> effect back then.
>
> > Could you explain what you mean by "edge coloring"?
>
> Traditional hand-drawn trees use different leaf or branch styles for
> the different lineages, this way you have visual clues that let you
> move in the tree.  One such tree with 400 individuals hangs on a wall
> in my living room.
>
> Edge coloring is an approximation to this using GraphViz.  Works by
> first determining the lineage of every individual.  You can use any
> criteria for building lineages but the most useful by far is the
> agnatic lineage, that is made up of all individuals descending by
> direct male lines from some person.  In many cases, western people in
> the same agnatic lineage share the same surname, but deviations are
> common.
>
> Then you assign a color to the most common lineages and make the edges
> joining them have that color.
>
> Another helpful hint is to make bold all edges from some distinguished
> person in the graph (you, one of your children, etc).  This way you
> can navigate easily from ancestors to descendants since you see easily
> what branch will lead you down and will not end in a cul-de-sac.
>
> I have code for all this (actually, it was included in the version of
> GraphViz.py I sent, but only for records).  I think I sent a sample
> graph with this just a few weeks ago.

How I think this kind of stuff should be done:
- There are separate files which implent following stuff for each output
  mode (stacked, normal, with family nodes) and which are imported by
  GraphViz plugin according to user selection:
  - Data classes containing just the information needed for each node
  - Methods for gathering that data from Gramps database into
    the data objects (which are hashed in dict and refer each other
    forming a graph)
  - Methods for post-processing the information in the node objects:
    - Gathering information to set the node and edge (node reference)
      attributes that cannot be decided by node properties alone (e.g.
      lineage stuff)
    - Sorting the references to other nodes
      (e.g. so that children are in birth order)
  - Functions outputting the data appropriately to GraphViz after it
    has been sorted
- GraphViz plugin implements default methods & data classes for these
  so that only stuff overriding the defaults (by inheriting) need to be
  written

The code is much cleaner and understandable if you collect the data
to a smart data structure and have separate data collection, manipulation
and output methods.

However, I don't have time to write this.  I can discuss it though.  :-)


> > I've always wondered about that, but for me RL always produces vertical
> > graph.  (dot v1.11 bug?)
>
> I checked this morning with version 2.6, but I think it has always
> been the same.  One orientation has descendants on the left and
> ancestors on the right, the other flips them.  To produce vertical
> progression, BT is needed.
>
> > I changed the rankdir to use BT and LR which work correctly for me.
> > BT and TB seemed to produce the same results.
>
> TB is not documented, I think, but it would not be the first
> undocumented option in GraphViz.
>
> > > I suggest using something in the line of the function implemented by
> > > the attached patch to NameDisplay.py.
> >
> > Could you send a similar patch against the attached version of
> > the GraphViz plugin?    The dates are now fetched in the
> > get_date_strings() method.
>
> The patch was not a GraphViz patch, but a NameDisplay patch, so that
> it can be refined for localization if needed.
>
> GraphViz.py only needs to call the new function if years-only is
> requested.

You mean GraphViz plugin could call the function from NameDisplay?


        - Eero


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=103432&bid=230486&dat=121642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Julio Sánchez-2
2006/2/12, Eero Tamminen <[hidden email]>:

> Hi,
>
> On Sunday 12 February 2006 00:52, Julio Sanchez wrote:
> ...
> No, but if you're already using command line, doing:
>         iconv -f iso-8859-1 -t utf-8 old.dot > new.dot
>         iconv -t utf-8 -f iso-8859-1 old.dot > new.dot
>
> Should be easy. :-)  Iconv is even mentioned in the GraphViz docs.
>
> I added note about this to the comment in the generated .dot file.
> Is there anything else that should be documented in the .dot file
> header?

How to do that on Windows or Mac, etc., for instance.

Really, it is a problem in Graphviz.  I think they know it and they
have been working on this.  I have inspected the source for 2.8 and
they seem to have it fixed in svggen.c, i.e.  SVG output no longer
cares about the input encoding.  If you feed it Latin1 and mark it so,
the output is identical, byte-by-byte, to that obtained from UTF-8
input.  I just checked on 2.6.

It is not working for PostScript yet, but I think it will.

> How I think this kind of stuff should be done:
> - There are separate files which implent following stuff for each output
>   mode (stacked, normal, with family nodes) and which are imported by
>   GraphViz plugin according to user selection:
>   - Data classes containing just the information needed for each node
>   - Methods for gathering that data from Gramps database into
>     the data objects (which are hashed in dict and refer each other
>     forming a graph)
>   - Methods for post-processing the information in the node objects:
>     - Gathering information to set the node and edge (node reference)
>       attributes that cannot be decided by node properties alone (e.g.
>       lineage stuff)
>     - Sorting the references to other nodes
>       (e.g. so that children are in birth order)
>   - Functions outputting the data appropriately to GraphViz after it
>     has been sorted
> - GraphViz plugin implements default methods & data classes for these
>   so that only stuff overriding the defaults (by inheriting) need to be
>   written
>
> The code is much cleaner and understandable if you collect the data
> to a smart data structure and have separate data collection, manipulation
> and output methods.
>
> However, I don't have time to write this.  I can discuss it though.  :-)

Interesting ideas.

> > The patch was not a GraphViz patch, but a NameDisplay patch, so that
> > it can be refined for localization if needed.
> >
> > GraphViz.py only needs to call the new function if years-only is
> > requested.
>
> You mean GraphViz plugin could call the function from NameDisplay?

Sorry for the confusion, DateDisplay is what I meant all the time.

Julio


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
<a href="http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642">http://sel.as-us.falkag.net/sel?cmd=lnk&kid3432&bid#0486&dat1642
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GraphViz plugin changes/improvements

Eero Tamminen-3
Hi,

On Monday 13 February 2006 00:49, Julio Sanchez wrote:

> > No, but if you're already using command line, doing:
> >         iconv -f iso-8859-1 -t utf-8 old.dot > new.dot
> >         iconv -t utf-8 -f iso-8859-1 old.dot > new.dot
> >
> > Should be easy. :-)  Iconv is even mentioned in the GraphViz docs.
> >
> > I added note about this to the comment in the generated .dot file.
> > Is there anything else that should be documented in the .dot file
> > header?
>
> How to do that on Windows or Mac, etc., for instance.
I wrote a Python script they could use (attached).


        - Eero

iconv.py (426 bytes) Download Attachment
12