Before gramps-4.1.3

classic Classic list List threaded Threaded
31 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Before gramps-4.1.3

jerome
Devs,


A wiki page[1] has been started for listing changes
on '4.1.3' since '4.1.2' version.

Still need to commit one fix for bug #8225
This can take a while since I make a choice ...

For fixing :

 File "/usr/lib/python3.4/site-packages/gramps/plugins/export/exportgeneweb.py", line 131, in write_family
    self.get_full_person_info_fam(mother)
  File "/usr/lib/python3.4/site-packages/gramps/plugins/export/exportgeneweb.py", line 86, in writeln
    self.g.write(self.iso8859('%s\n' % (text)))
TypeError: must be str, not bytes

We have two contributions !

diff --git a/gramps/plugins/export/exportgeneweb.py b/gramps/plugins/export/exportgeneweb.py
index 175a5e1..daa4897 100644
--- a/gramps/plugins/export/exportgeneweb.py
+++ b/gramps/plugins/export/exportgeneweb.py
@@ -30,6 +30,7 @@
 #
 #-------------------------------------------------------------------------
 import os
+import sys
 
 #------------------------------------------------------------------------
 #
@@ -481,6 +482,8 @@ class GeneWebWriter(object):
         return retval
 
     def iso8859(self,s):
+        if sys.version_info[0] >= 3:
+            return s.encode('iso-8859-1','xmlcharrefreplace').decode('iso-8859-1')
         return s.encode('iso-8859-1','xmlcharrefreplace')
 
 #-------------------------------------------------------------------------

or

diff --git a/gramps/plugins/export/exportgeneweb.py b/gramps/plugins/export/exportgeneweb.py
index 175a5e1..5cc4730 100644
--- a/gramps/plugins/export/exportgeneweb.py
+++ b/gramps/plugins/export/exportgeneweb.py
@@ -89,7 +89,7 @@ class GeneWebWriter(object):
 
         self.dirname = os.path.dirname (self.filename)
         try:
- self.g = open(self.filename, "w")
+ self.g = open(self.filename, "wb")
         except IOError as msg:
             msg2 = _("Could not create %s") % self.filename
             self.user.notify_error(msg2, str(msg))


I do not yet test both solutions, but at a glance, they all seem to be right.

The second one sounds a little bit more logical (for me), but I do not
know if this should be the solution...



[1] https://gramps-project.org/wiki/index.php?title=Template:Releases/4.1.3
[2] https://gramps-project.org/bugs/view.php?id=8225


regards,
Jérôme

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Before gramps-4.1.3

enno
Jerome,

> Still need to commit one fix for bug #8225
> This can take a while since I make a choice ...
>
> For fixing :
>
>   File "/usr/lib/python3.4/site-packages/gramps/plugins/export/exportgeneweb.py", line 131, in write_family
>      self.get_full_person_info_fam(mother)
>    File "/usr/lib/python3.4/site-packages/gramps/plugins/export/exportgeneweb.py", line 86, in writeln
>      self.g.write(self.iso8859('%s\n' % (text)))
> TypeError: must be str, not bytes
>
> We have two contributions !
>
> diff --git a/gramps/plugins/export/exportgeneweb.py b/gramps/plugins/export/exportgeneweb.py
> index 175a5e1..daa4897 100644
> --- a/gramps/plugins/export/exportgeneweb.py
> +++ b/gramps/plugins/export/exportgeneweb.py
> @@ -30,6 +30,7 @@
>   #
>   #-------------------------------------------------------------------------
>   import os
> +import sys
>  
>   #------------------------------------------------------------------------
>   #
> @@ -481,6 +482,8 @@ class GeneWebWriter(object):
>           return retval
>  
>       def iso8859(self,s):
> +        if sys.version_info[0] >= 3:
> +            return s.encode('iso-8859-1','xmlcharrefreplace').decode('iso-8859-1')
>           return s.encode('iso-8859-1','xmlcharrefreplace')
>  
>   #-------------------------------------------------------------------------
>
> or
>
> diff --git a/gramps/plugins/export/exportgeneweb.py b/gramps/plugins/export/exportgeneweb.py
> index 175a5e1..5cc4730 100644
> --- a/gramps/plugins/export/exportgeneweb.py
> +++ b/gramps/plugins/export/exportgeneweb.py
> @@ -89,7 +89,7 @@ class GeneWebWriter(object):
>  
>           self.dirname = os.path.dirname (self.filename)
>           try:
> - self.g = open(self.filename, "w")
> + self.g = open(self.filename, "wb")
>           except IOError as msg:
>               msg2 = _("Could not create %s") % self.filename
>               self.user.notify_error(msg2, str(msg))
>
>
> I do not yet test both solutions, but at a glance, they all seem to be right.
>
> The second one sounds a little bit more logical (for me), but I do not
> know if this should be the solution...
I agree. If GeneWeb alows iso-8859, the file must be written as binary.
It's also a solution which is probably well tested by the user who
suggested it, i.e. with a working GeneWeb installation.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Before gramps-4.1.3

jerome
Enno,

> "I agree. If GeneWeb alows iso-8859, the file must be written as binary.
It's also a solution which is probably well tested by the user who
suggested it, i.e. with a working GeneWeb installation."

I hope so; only supposing that GeneWeb is still able to convert iso-8859!
https://github.com/geneanet/geneweb/blob/master/src/mutil.ml
https://github.com/geneanet/geneweb/blob/master/gwb2ged/gwb2ged.ml
etc...

True, maybe a .gw file format[1] review for gramps 4.2.
But that's something else.

Thanks!

[1] https://gramps-project.org/bugs/view.php?id=3094

--------------------------------------------
En date de : Mer 29.4.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] Before gramps-4.1.3
 À: [hidden email]
 Date: Mercredi 29 avril 2015, 15h45
 
 Jerome,
 > Still need to commit one fix for bug
 #8225
 > This can take a while since I
 make a choice ...
 >
 >
 For fixing :
 >
 >   File
 "/usr/lib/python3.4/site-packages/gramps/plugins/export/exportgeneweb.py",
 line 131, in write_family
 >     
 self.get_full_person_info_fam(mother)
 > 
   File
 "/usr/lib/python3.4/site-packages/gramps/plugins/export/exportgeneweb.py",
 line 86, in writeln
 >     
 self.g.write(self.iso8859('%s\n' % (text)))
 > TypeError: must be str, not bytes
 >
 > We have two
 contributions !
 >
 >
 diff --git a/gramps/plugins/export/exportgeneweb.py
 b/gramps/plugins/export/exportgeneweb.py
 > index 175a5e1..daa4897 100644
 > ---
 a/gramps/plugins/export/exportgeneweb.py
 > +++
 b/gramps/plugins/export/exportgeneweb.py
 > @@ -30,6 +30,7 @@
 >   #
 >   #-------------------------------------------------------------------------
 >   import os
 > +import sys
 >   
 >   #------------------------------------------------------------------------
 >   #
 > @@
 -481,6 +482,8 @@ class GeneWebWriter(object):
 >           return retval
 >   
 >   
    def iso8859(self,s):
 > + 
       if sys.version_info[0] >= 3:
 > +            return
 s.encode('iso-8859-1','xmlcharrefreplace').decode('iso-8859-1')
 >           return
 s.encode('iso-8859-1','xmlcharrefreplace')
 >   
 >   #-------------------------------------------------------------------------
 >
 > or
 >
 > diff --git
 a/gramps/plugins/export/exportgeneweb.py
 b/gramps/plugins/export/exportgeneweb.py
 > index 175a5e1..5cc4730 100644
 > ---
 a/gramps/plugins/export/exportgeneweb.py
 > +++
 b/gramps/plugins/export/exportgeneweb.py
 > @@ -89,7 +89,7 @@ class
 GeneWebWriter(object):
 >   
 >           self.dirname =
 os.path.dirname (self.filename)
 >     
      try:
 > - self.g =
 open(self.filename, "w")
 > +
 self.g = open(self.filename, "wb")
 >           except IOError as
 msg:
 >           
    msg2 = _("Could not create %s") %
 self.filename
 >           
    self.user.notify_error(msg2, str(msg))
 >
 >
 >
 I do not yet test both solutions, but at a glance, they all
 seem to be right.
 >
 >
 The second one sounds a little bit more logical (for me),
 but I do not
 > know if this should be the
 solution...
 I agree. If GeneWeb alows
 iso-8859, the file must be written as binary.
 It's also a solution which is probably well
 tested by the user who
 suggested it, i.e.
 with a working GeneWeb installation.
 
 regards,
 
 Enno
 
 
 ------------------------------------------------------------------------------
 One dashboard for servers and applications
 across Physical-Virtual-Cloud
 Widest
 out-of-the-box monitoring support with 50+ applications
 Performance metrics, stats and reports that
 give you Actionable Insights
 Deep dive
 visibility with transaction tracing using APM Insight.
 http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
 _______________________________________________
 Gramps-devel mailing list
 [hidden email]
 https://lists.sourceforge.net/lists/listinfo/gramps-devel
 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Before gramps-4.1.3

enno
Jerome,

> I hope so; only supposing that GeneWeb is still able to convert iso-8859!
> https://github.com/geneanet/geneweb/blob/master/src/mutil.ml
> https://github.com/geneanet/geneweb/blob/master/gwb2ged/gwb2ged.ml
> etc...

> True, maybe a .gw file format[1] review for gramps 4.2.
> But that's something else.

> [1] https://gramps-project.org/bugs/view.php?id=3094

I guess it won't be easy without more details from the GeneWeb side, like more detailed error information.

I uploaded a .gw file to GeneaNet today, exported from 3.4.9, and got a message that there's something wrong with my GEDCOM file! No further details other than that the file is invalid, i.e. incorrect extension or damaged.

I haven't tested with a GeneWeb server on Linux or Windows, but with this lack of information, I don't see many ways to improve in Gramps.

Do you have any contacts on GeneaNet? It's a popular site in Europe, and uploads in GeneWeb format may be better than GEDCOM, if they work.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

GeneWeb [was Before gramps-4.1.3]

jerome
Enno,

Yes, hard to find a complete documentation
about the geneweb file format.

I planned to make a quick 'diff' between current
.gw file generation by Gramps and .gw file
generation by Geneweb via our 'example.gramps'[1].

I did not plan to test Geneweb 7 yet (alpha state).

By contacts, do you mean on the geneanet team,
or distant cousins linked via Geneanet accounts?

Some years ago, I "marked" cousins having common
ancestors. Now, I sometimes contact them, but most
part of time, they ask more informations.
I removed my tree because it was really difficult
to maintain it and they changed the privacy policy.
It was (is?) possible to keep an index (surname, place)
for searches.

About the geneanet team, I had one or two contact(s)
in the past. eg, http://www.galichon.com/
https://www.linkedin.com/pub/jean-yves-baxter/57/b64/137
we might find more. eg,
http://fr.viadeo.com/fr/company/geneanet


[1] https://gramps-project.org/bugs/view.php?id=3094


regards,

Jérôme

--------------------------------------------
En date de : Sam 2.5.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] Before gramps-4.1.3
 À: "jerome" <[hidden email]>, [hidden email]
 Date: Samedi 2 mai 2015, 17h11
 
 Jerome,
 
 > I hope so; only supposing
 that GeneWeb is still able to convert iso-8859!
 > https://github.com/geneanet/geneweb/blob/master/src/mutil.ml
 > https://github.com/geneanet/geneweb/blob/master/gwb2ged/gwb2ged.ml
 > etc...
 
 > True, maybe a .gw file format[1] review
 for gramps 4.2.
 > But that's
 something else.
 
 > [1] https://gramps-project.org/bugs/view.php?id=3094
 
 I guess it won't be easy
 without more details from the GeneWeb side, like more
 detailed error information.
 
 I uploaded a .gw file to GeneaNet today,
 exported from 3.4.9, and got a message that there's
 something wrong with my GEDCOM file! No further details
 other than that the file is invalid, i.e. incorrect
 extension or damaged.
 
 I
 haven't tested with a GeneWeb server on Linux or
 Windows, but with this lack of information, I don't see
 many ways to improve in Gramps.
 
 Do you have any contacts on GeneaNet? It's
 a popular site in Europe, and uploads in GeneWeb format may
 be better than GEDCOM, if they work.
 
 regards,
 
 Enno
 
 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

Eric Doutreleau
There s a draft about the new format of geneanet based on gwplus format.
http://opensource.geneanet.org/issues/333

Right now it s better to use gedcom to upload data to geneanet instead
of the geneweb output.

On 03/05/2015 10:59, jerome wrote:

> Enno,
>
> Yes, hard to find a complete documentation
> about the geneweb file format.
>
> I planned to make a quick 'diff' between current
> .gw file generation by Gramps and .gw file
> generation by Geneweb via our 'example.gramps'[1].
>
> I did not plan to test Geneweb 7 yet (alpha state).
>
> By contacts, do you mean on the geneanet team,
> or distant cousins linked via Geneanet accounts?
>
> Some years ago, I "marked" cousins having common
> ancestors. Now, I sometimes contact them, but most
> part of time, they ask more informations.
> I removed my tree because it was really difficult
> to maintain it and they changed the privacy policy.
> It was (is?) possible to keep an index (surname, place)
> for searches.
>
> About the geneanet team, I had one or two contact(s)
> in the past. eg, http://www.galichon.com/
> https://www.linkedin.com/pub/jean-yves-baxter/57/b64/137
> we might find more. eg,
> http://fr.viadeo.com/fr/company/geneanet
>
>
> [1] https://gramps-project.org/bugs/view.php?id=3094
>
>
> regards,
>
> Jérôme
>
> --------------------------------------------
> En date de : Sam 2.5.15, Enno Borgsteede <[hidden email]> a écrit :
>
>   Objet: Re: [Gramps-devel] Before gramps-4.1.3
>   À: "jerome" <[hidden email]>, [hidden email]
>   Date: Samedi 2 mai 2015, 17h11
>  
>   Jerome,
>  
>   > I hope so; only supposing
>   that GeneWeb is still able to convert iso-8859!
>   > https://github.com/geneanet/geneweb/blob/master/src/mutil.ml
>   > https://github.com/geneanet/geneweb/blob/master/gwb2ged/gwb2ged.ml
>   > etc...
>  
>   > True, maybe a .gw file format[1] review
>   for gramps 4.2.
>   > But that's
>   something else.
>  
>   > [1] https://gramps-project.org/bugs/view.php?id=3094
>  
>   I guess it won't be easy
>   without more details from the GeneWeb side, like more
>   detailed error information.
>  
>   I uploaded a .gw file to GeneaNet today,
>   exported from 3.4.9, and got a message that there's
>   something wrong with my GEDCOM file! No further details
>   other than that the file is invalid, i.e. incorrect
>   extension or damaged.
>  
>   I
>   haven't tested with a GeneWeb server on Linux or
>   Windows, but with this lack of information, I don't see
>   many ways to improve in Gramps.
>  
>   Do you have any contacts on GeneaNet? It's
>   a popular site in Europe, and uploads in GeneWeb format may
>   be better than GEDCOM, if they work.
>  
>   regards,
>  
>   Enno
>  
>  
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Gramps-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

enno
In reply to this post by jerome
Jerome,
> By contacts, do you mean on the geneanet team,
> or distant cousins linked via Geneanet accounts?
I mean the GeneaNet or GeneWeb team, because I fear that when I contact
normal help, I will be treated like an average consumer, and will be
asked to upload another file, or GEDCOM, where as a developer, I really
want to know what's wrong with the GW file, like illegal characters,
extra fields that GeneaNet does not recognize, like that.

So, what I'm hoping for is that someone can look into the server log
files, and give some technical information about the errors in my file.
Right now, I can see that the upload works, but the parsing fails, and
that's it.

The alternative would be to find out which GeneWeb version is used on
the GeneaNet site, and test with that. My normal uploads in GEDCOM
format work, so I my tree uploads are not really affected, and I'm not
interested in running GeneWeb on my own site or PC either. It's just for
Gramps debug purposes.

I have no idea how many actually use GW export, and whether it makes
sense to keep it as a main module. It's useful for GeneaNet, or for
upload to a GeneWeb site of your own, but how much is it used world
wide, compared to other format? I have no idea.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

jerome
In reply to this post by jerome
Enno,


I ran a local version of GeneWeb 6.07, which should be the same as
under geneanet servers. My testing .gw file was generated from gramps-master
(python3) and except some warnings into log, the content has been imported
into geneweb.

$ cd "geneweb-6.07/bases"
$ ../gw/gwc -nofail -f Gramps/example/gramps/Untitled_1.gw -o test -stats -cg > comm.log

File "../Gramps/example/gramps/Untitled_1.gw", line 4:
Error: fam Wong .2 0 0 + Page Mildred.3
File "../Gramps/example/gramps/Untitled_1.gw", line 5:
Error: src Import_from_test2.ged
File "../Gramps/example/gramps/Untitled_1.gw", line 25:
Error: fam &#1052;&#1072;&#1088;&#1090;&#1099;&#1085;&#1086;&#1074; .16 0 0 + Reed Jane.17
File "../Gramps/example/gramps/Untitled_1.gw", line 26:
Error: src Import_from_test2.ged
File "../Gramps/example/gramps/Untitled_1.gw", line 28:
Error: fam Harrison Paul_Allen.18 26/10/1958  +16/10/1983 &#1044;&#1072;&#1085;&#1080;&#1083;&#1086;&#1074; Lucinda_Elinor.19
File "../Gramps/example/gramps/Untitled_1.gw", line 29:
Error: src Import_from_test2.ged
File "../Gramps/example/gramps/Untitled_1.gw", line 30:
Error: beg
File "../Gramps/example/gramps/Untitled_1.gw", line 31:
Error: - h Benjamin_Allen.20 10/7/1986  
...
Error: src Import_from_test2.ged
File "../Gramps/example/gramps/Untitled_1.gw", line 4202:
Error: fam Lessard .1944  #bp Waterloo-Cedar_Falls,_IA 0 + Castro ???.697
File "../Gramps/example/gramps/Untitled_1.gw", line 4203:
Error: src Import_from_test2.ged
File "../Gramps/example/gramps/Untitled_1.gw", line 4204:
Error: beg
File "../Gramps/example/gramps/Untitled_1.gw", line 4205:
Error: - h Isaac.1370 10/1828 #bp Mountain_Home,_ID 0
File "../Gramps/example/gramps/Untitled_1.gw", line 4206:
Error: end
...
Warning: Rufus.868 Blanco was parent at age of 1836
...
Warning: Luella Jacques.208 Martel
  is born after the death of his/her mother
Ruth Ann.1034 Hébert
...
Warning: Christina.1066 Lucas born after his/her child Rufus.868 Blanco
...
Warning: Changed order of marriages of Eleanor Irene.1099 Page
...
Warning: The following children of
  William M..1576 Rodriquez
and
  Mary"Polly".1577 Douglas
are born very close:
- William Frederick.1578 Rodriquez
- John.1579 Rodriquez
...
Warning: Major Marquis II.111 Brooks
  is born after the death of his/her mother
Isabella.1616 Guzman
...
Warning: The difference of age between Cyrus.1618 Morris and Martha.1619 Graves is quite important: 1778
...
Warning: Christina.1627 Norton married at age 0
...
Warning: The following children of
  Eugene Stanley.209 Garner
and
  Frances Lucille (Babe).878 Reed
are not in order:
- Howard Lane.1762 Garner
- John Roger.590 Garner
...
Warning: Shirley Kay.1598 Warner was parent at age of 10
...
Warning: Changed order of children of Benjamin H..1818 Farmer and Isabella.1932 Mills
...
Undefined: Mary Ann.38 Delgado
Undefined: Anna June.57 Kristensen
Undefined: Benjamin.73 Allen
...
790 men
720 women
4 unknown sex
0 unnamed
Oldest: Rufus.868 Blanco, 1846
Oldest still alive: Hannah.462 Kowalski, 425
Youngest father: Jason Richard.1589 Garner, 18
Youngest mother: Christina.1066 Lucas, -1760
Oldest father: Rufus.868 Blanco, 1836
Oldest mother: Helewisa.981 Schwartz, 82

The  content of the log is cryptic but I was able to create the base.

Yes, maybe something rejected (strict) on geneanet side.
Will try without the '-nofail' flag on gwc.

$ gwc -nofail


Well, maybe I will look at .gw import from gramps
before an export?


regards,

Jérôme


--------------------------------------------
En date de : Dim 3.5.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: GeneWeb [was Before gramps-4.1.3]
 À: "jerome" <[hidden email]>, [hidden email]
 Date: Dimanche 3 mai 2015, 20h08
 
 Jerome,
 > By contacts, do you mean on the geneanet
 team,
 > or distant cousins linked via
 Geneanet accounts?
 I mean the GeneaNet or
 GeneWeb team, because I fear that when I contact
 normal help, I will be treated like an average
 consumer, and will be
 asked to upload
 another file, or GEDCOM, where as a developer, I really
 want to know what's wrong with the GW file,
 like illegal characters,
 extra fields that
 GeneaNet does not recognize, like that.
 
 So, what I'm hoping for is that someone can
 look into the server log
 files, and give
 some technical information about the errors in my file.
 Right now, I can see that the upload works, but
 the parsing fails, and
 that's it.
 
 The alternative would be to
 find out which GeneWeb version is used on
 the GeneaNet site, and test with that. My
 normal uploads in GEDCOM
 format work, so I
 my tree uploads are not really affected, and I'm not
 interested in running GeneWeb on my own site or
 PC either. It's just for
 Gramps debug
 purposes.
 
 I have no idea
 how many actually use GW export, and whether it makes
 sense to keep it as a main module. It's
 useful for GeneaNet, or for
 upload to a
 GeneWeb site of your own, but how much is it used world
 wide, compared to other format? I have no
 idea.
 
 regards,
 
 Enno
 


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

enno
Jerome,

> I ran a local version of GeneWeb 6.07, which should be the same as
> under geneanet servers. My testing .gw file was generated from gramps-master
> (python3) and except some warnings into log, the content has been imported
> into geneweb.
>
> $ cd "geneweb-6.07/bases"
> $ ../gw/gwc -nofail -f Gramps/example/gramps/Untitled_1.gw -o test -stats -cg > comm.log
>
> File "../Gramps/example/gramps/Untitled_1.gw", line 4:
> Error: fam Wong .2 0 0 + Page Mildred.3
> File "../Gramps/example/gramps/Untitled_1.gw", line 5:
> Error: src Import_from_test2.ged
After reading this, I did a similar thing on Linux, and found the same
cryptic error messages like you. And with those, I'm not going to spend
more time on this, because it would be a lot of trial and error, and
like Eric wrote, import from GEDCOM works OK.

The GeneWeb data model is quite primitive, and one of the limitations
that it seems to have is that it only allows a single source per event,
and of that source, the only thing that it can store is the source
title. In other words, exporting means to GeneWeb means such a huge data
loss, that I don't see any advantage over GEDCOM at all. There's more
things that it doesn't seem to support, like multiple parent families,
which I have in my tree a few times, for a good reason, so why bother?

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

Eric Doutreleau
Hi

Enno i agree with you.
As there s no ( yet ) official description of the gw format it will be a
lot of trial and errors.
in the link i give before there a kind of draft of description.

Nevertheless for geneanet usrs there could be some advantages to use
gwplus format than gedcom.
for example concerning testimonials on a event.
they are lost through the export of the gedcom
and they could be kepy if we use the gwplus format.

i have begun to study a bit the format but as my skills in python are
not very good i need a lot of work to improve
the export.

On 04/05/2015 22:33, Enno Borgsteede wrote:

> Jerome,
>> I ran a local version of GeneWeb 6.07, which should be the same as
>> under geneanet servers. My testing .gw file was generated from gramps-master
>> (python3) and except some warnings into log, the content has been imported
>> into geneweb.
>>
>> $ cd "geneweb-6.07/bases"
>> $ ../gw/gwc -nofail -f Gramps/example/gramps/Untitled_1.gw -o test -stats -cg > comm.log
>>
>> File "../Gramps/example/gramps/Untitled_1.gw", line 4:
>> Error: fam Wong .2 0 0 + Page Mildred.3
>> File "../Gramps/example/gramps/Untitled_1.gw", line 5:
>> Error: src Import_from_test2.ged
> After reading this, I did a similar thing on Linux, and found the same
> cryptic error messages like you. And with those, I'm not going to spend
> more time on this, because it would be a lot of trial and error, and
> like Eric wrote, import from GEDCOM works OK.
>
> The GeneWeb data model is quite primitive, and one of the limitations
> that it seems to have is that it only allows a single source per event,
> and of that source, the only thing that it can store is the source
> title. In other words, exporting means to GeneWeb means such a huge data
> loss, that I don't see any advantage over GEDCOM at all. There's more
> things that it doesn't seem to support, like multiple parent families,
> which I have in my tree a few times, for a good reason, so why bother?
>
> regards,
>
> Enno
>
>
> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________
> Gramps-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

enno
Hi Eric,
> As there s no ( yet ) official description of the gw format it will be a
> lot of trial and errors.
> in the link i give before there a kind of draft of description.
I read that, and it's not much yet, but I hope it will be improved.
> Nevertheless for geneanet usrs there could be some advantages to use
> gwplus format than gedcom.
> for example concerning testimonials on a event.
You mean witnesses, right? I saw wit mentioned in the draft.
> i have begun to study a bit the format but as my skills in python are
> not very good i need a lot of work to improve
> the export.
Well, if you run a local GeneWeb like Jerome mentioned, and I tried too,
it's a nice way to learn ...

And although my opinion of GeneWeb is that it's quite primitive, with
sources etc., I understand its importance for GeneaNet, and I like
GeneaNet because it's large, multi-lingual, and not as commercial as My
Heritage (and Ancestry).

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

jerome
Enno,

I can just suppose that the poor source support on geneweb has some
historical causes.

Geneweb file format (.gw), as a family centric design, cannot properly handle
multiple sources. Also, gedcom support was a basic one (title on some
records like source or place).

I did not know there was a gwplus (or gw2!) file format!

Note, the current design seems also logical.
eg, on http://roglo.eu/roglo?lang=nl;i=1023996
we can see that references into notes are
like citations (bibliographie, notices, notities).

So, the .gw file format is keeping basic source (title, ref),
but is also including internal citations on notes.

What is bother me, was maybe the internal sources dictionnary
into geneweb! I saw it during navigation, then never be able to get it back

Note, during investigations around geneweb file format, I thought
that comma was allowed as a separator for multiple sources (titles)?

I like the ability to merge multiple .gw files (bases),
but I dislike the family centric design[1]. The internal places
and sources "tables/dictionnaries" are maybe oriented "geneanet",
like the missing repository support and the media handling, but
as the local version is available and as the geneweb file
format is open (not documented ...still open document ?),
maybe to still (try to) support this file format is a good feature.

[1] https://gramps-project.org/bugs/view.php?id=2488


Jérôme


--------------------------------------------
En date de : Mar 5.5.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] GeneWeb [was Before gramps-4.1.3]
 À: [hidden email]
 Date: Mardi 5 mai 2015, 15h58
 
 Hi Eric,
 > As there s no ( yet ) official description
 of the gw format it will be a
 > lot of
 trial and errors.
 > in the link i give
 before there a kind of draft of description.
 I read that, and it's not much yet, but I
 hope it will be improved.
 > Nevertheless
 for geneanet usrs there could be some advantages to use
 > gwplus format than gedcom.
 > for example concerning testimonials on a
 event.
 You mean witnesses, right? I saw wit
 mentioned in the draft.
 > i have begun to
 study a bit the format but as my skills in python are
 > not very good i need a lot of work to
 improve
 > the export.
 Well, if you run a local GeneWeb like Jerome
 mentioned, and I tried too,
 it's a nice
 way to learn ...
 
 And
 although my opinion of GeneWeb is that it's quite
 primitive, with
 sources etc., I understand
 its importance for GeneaNet, and I like
 GeneaNet because it's large, multi-lingual,
 and not as commercial as My
 Heritage (and
 Ancestry).
 
 regards,
 
 Enno
 
 
 ------------------------------------------------------------------------------
 One dashboard for servers and applications
 across Physical-Virtual-Cloud
 Widest
 out-of-the-box monitoring support with 50+ applications
 Performance metrics, stats and reports that
 give you Actionable Insights
 Deep dive
 visibility with transaction tracing using APM Insight.
 http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
 _______________________________________________
 Gramps-devel mailing list
 [hidden email]
 https://lists.sourceforge.net/lists/listinfo/gramps-devel
 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

jerome
In reply to this post by jerome
Enno, Eric,

Looks like gramps already deprecates .gw file format support
(geneweb 4?) on import, and might try to support gwplus (geneweb 5? 6, 7?)?
It is right that the new geneweb file format sounds much better.

Thanks to logging and recent parser review on import geneweb module.
Debug is simplier when we get these complete LOG infos. :-)

event records : personal events, family events, etc ...
outside a top level like family (or person), is close to gramps' way.

It takes me few minutes for generating a quick draft (proof) for
the import. We could easily translate these events with gramps types
and the parsing should be more consistent.
I did not look at way, which is linking these events to person or family
(could be really the name as key ???), but it seems we
could provide a better geneweb support, at least on import.


Jérôme

--------------------------------------------
En date de : Lun 4.5.15, Eric Doutreleau <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] GeneWeb [was Before gramps-4.1.3]
 À: [hidden email]
 Date: Lundi 4 mai 2015, 23h37
 
 Hi
 
 Enno i agree with you.
 As there
 s no ( yet ) official description of the gw format it will
 be a
 lot of trial and errors.
 in the link i give before there a kind of draft
 of description.
 
 Nevertheless for geneanet usrs there could be
 some advantages to use
 gwplus format than
 gedcom.
 for example concerning testimonials
 on a event.
 they are lost through the export
 of the gedcom
 and they could be kepy if we
 use the gwplus format.
 
 i
 have begun to study a bit the format but as my skills in
 python are
 not very good i need a lot of
 work to improve
 the export.
 
 ------------------------------------------------------------------------------
 > One dashboard for servers and applications
 across Physical-Virtual-Cloud
 > Widest
 out-of-the-box monitoring support with 50+ applications
 > Performance metrics, stats and reports
 that give you Actionable Insights
 > Deep
 dive visibility with transaction tracing using APM
 Insight.
 > http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
 >
 _______________________________________________
 > Gramps-devel mailing list
 > [hidden email]
 > https://lists.sourceforge.net/lists/listinfo/gramps-devel
 
 
 ------------------------------------------------------------------------------
 One dashboard for servers and applications
 across Physical-Virtual-Cloud
 Widest
 out-of-the-box monitoring support with 50+ applications
 Performance metrics, stats and reports that
 give you Actionable Insights
 Deep dive
 visibility with transaction tracing using APM Insight.
 http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
 _______________________________________________
 Gramps-devel mailing list
 [hidden email]
 https://lists.sourceforge.net/lists/listinfo/gramps-devel
------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel

8548.patch (8K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

enno
In reply to this post by jerome
Jerome,
> I can just suppose that the poor source support on geneweb has some
> historical causes.
I agree. It happens in a lot of programs, and depends on author's
preferences. Popular Dutch programs like Aldfaer en Pro-Gen have the
same simple support, which works quite well for disciplined users, who
can simply type a formatted citation text.

> Note, the current design seems also logical.
> eg, on http://roglo.eu/roglo?lang=nl;i=1023996
> we can see that references into notes are
> like citations (bibliographie, notices, notities).
Well, it looks more advanced than it actually is. I can see that,
because you sent a Dutch language URL, which shows the notes heading in
Dutch, as Notities. The contents however are in French with Biographie,
which I would spell as Biografie, and the same goes for Notes et
références, which is also not generated by GeneWeb itself. What you see
here looks like html formatted text copied from wikipedia page
referenced at the bottom, under the Bronnen (sources) heading. If you
follow the link in there, you can see what I mean.
> So, the .gw file format is keeping basic source (title, ref),
> but is also including internal citations on notes.
No. It seems to be keeping some of the html copied from the wikipedia
page, but it's incomplete. You can see that, because the footnote links
[1], [2], [3], etc. don't work, which is the exact same behavior that I
get when I copy portions of a wikipedia page to notes in Gramps. I can't
see how italic, bold, and horizontal lines were coded here, but that may
be simple html that GeneWeb allow in a note, and keeps when rendering a
page.

> What is bother me, was maybe the internal sources dictionnary
> into geneweb! I saw it during navigation, then never be able to get it back
How do you mean bother? I can see one when I access my GeneaNet tree in
admin mode, but it doesn't seem to be visible for visitors.
> Note, during investigations around geneweb file format, I thought
> that comma was allowed as a separator for multiple sources (titles)?
I hope not, because comma is a common used separator between author,
title, subtitle, page, etc.

> I like the ability to merge multiple .gw files (bases),
> but I dislike the family centric design[1]. The internal places
> and sources "tables/dictionnaries" are maybe oriented "geneanet",
> like the missing repository support and the media handling, but
> as the local version is available and as the geneweb file
> format is open (not documented ...still open document ?),
> maybe to still (try to) support this file format is a good feature.
>
> [1] https://gramps-project.org/bugs/view.php?id=2488
I agree, but it's quite common on the web. I have a few cousins who are
not really cousins because their biological father is not the registered
one, and GeneaNet only shows the biological one. Apparently, when
importing from GEDCOM, GeneWeb shows the other one away, and maybe these
cousins are a cause for gw import going wrong. Sites like Ancestry, Geni
and My Heritage have similar restrictions, and on Ancestry and My
Heritage, my GEDCOM imports simply fail. Ancestry gives me a useless
error message, like GeneaNet does with gw import, and on My Heritage the
import simply quits, meaning that they promise to send an email when the
import is done, and that email never comes.

If we can afford it, I think that supporting the gw format is a good
idea indeed, because GeneaNet is big on the European continent. We may
need to use plug-ins though, because GeneWeb software downloads are
still version 6, mentioning 7 as alpha, and GeneaNet already runs on 7!

I just downloaded my tree from GeneaNet to run a simple test, and plan
to report about that later.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

jerome
Enno,

> > Note, the
 current design seems also logical.
 > > eg,
 on http://roglo.eu/roglo?lang=nl;i=1023996
 > > we can see that references into notes
 are
 > > like citations (bibliographie,
 notices, notities).
 > Well, it looks more
 advanced than it actually is.

Also, this page has been generated with the version
5.02 ... And on Eric's feature request[1], we can see
that they are trying to clear this (convention?):

"DataBaseNotes # start token is notes-db
ExtendedPages # start token is pages-ext

WizardNotes # start token is wizard-note # What are Wizard note? Only one? Last of sequence?

[..]

3/ PersonalNotes

Same as gw 5.00

notes LastNAme FirstNBame[.Number]
beg
Notes go there in a free format
end notes

[..]

7/ ExtendedPages

    extended page "PageName" used by:
    - person "PersonFirstName.Number PersonLastName" # multiple lines indicating usage by Person
    - extended page "PageName" # or by other ExtendedPage
    page-ext PageName
    Free text (two spaces at the beginning of each line)
    Continued free text
    end page-ext"

Sure, it sounds simple html, logical for gen web!

> How do you mean bother? I can
 see one when I access my GeneaNet tree in
 admin mode, but it doesn't seem to be
 visible for visitors.

Well, on local, I did not see this list anymore!
Ah yes, maybe should be logged as wizard...

> > Note, during
 investigations around geneweb file format, I thought
> > that comma was allowed as a separator for
 multiple sources (titles)?
 > I hope not,
 because comma is a common used separator between author,
 title, subtitle, page, etc.

At a glance, these "secondary fields" are using bold markup under
my testing file! So, layout versus data process?
As said, comma as separator was maybe an edition policy, but
it seems to work into one source title(s) field!

> > GeneWeb shows the other one away, and maybe
 these cousins are a cause for gw import
 going wrong. Sites like Ancestry, Geni
 and My Heritage have similar restrictions, and on Ancestry and
 My Heritage, my GEDCOM imports simply fail.

Yes, family section on gwplus spec is maybe still unclear[2] ...
Well, looks like gramps also has such limitations[3]!
That's maybe a common limitations?

> > GeneaNet already runs on 7!

I thought I used 6.07 some weeks ago for my export.
Ah, then "gwplus = version 7", right?

[1] http://opensource.geneanet.org/issues/333
[2] http://opensource.geneanet.org/issues/333#note-3
[3] https://gramps-project.org/wiki/index.php?title=GEPS_001:_Relationship_type_event_link


regards,
Jérôme


--------------------------------------------
En date de : Jeu 7.5.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] GeneWeb [was Before gramps-4.1.3]
 À: "jerome" <[hidden email]>, [hidden email]
 Date: Jeudi 7 mai 2015, 14h36
 
 Jerome,
 > I can just suppose that the poor source
 support on geneweb has some
 > historical
 causes.
 I agree. It happens in a lot of
 programs, and depends on author's
 preferences. Popular Dutch programs like
 Aldfaer en Pro-Gen have the
 same simple
 support, which works quite well for disciplined users, who
 
 can simply type a formatted citation
 text.
 
 > Note, the
 current design seems also logical.
 > eg,
 on http://roglo.eu/roglo?lang=nl;i=1023996
 > we can see that references into notes
 are
 > like citations (bibliographie,
 notices, notities).
 Well, it looks more
 advanced than it actually is. I can see that,
 because you sent a Dutch language URL, which
 shows the notes heading in
 Dutch, as
 Notities. The contents however are in French with
 Biographie,
 which I would spell as
 Biografie, and the same goes for Notes et
 références, which is also not generated by
 GeneWeb itself. What you see
 here looks
 like html formatted text copied from wikipedia page
 referenced at the bottom, under the Bronnen
 (sources) heading. If you
 follow the link
 in there, you can see what I mean.
 > So,
 the .gw file format is keeping basic source (title, ref),
 > but is also including internal citations
 on notes.
 No. It seems to be keeping some of
 the html copied from the wikipedia
 page,
 but it's incomplete. You can see that, because the
 footnote links
 [1], [2], [3], etc.
 don't work, which is the exact same behavior that I
 get when I copy portions of a wikipedia page to
 notes in Gramps. I can't
 see how
 italic, bold, and horizontal lines were coded here, but that
 may
 be simple html that GeneWeb allow in a
 note, and keeps when rendering a
 page.
 
 > What is bother me, was
 maybe the internal sources dictionnary
 >
 into geneweb! I saw it during navigation, then never be able
 to get it back
 How do you mean bother? I can
 see one when I access my GeneaNet tree in
 admin mode, but it doesn't seem to be
 visible for visitors.
 > Note, during
 investigations around geneweb file format, I thought
 > that comma was allowed as a separator for
 multiple sources (titles)?
 I hope not,
 because comma is a common used separator between author,
 title, subtitle, page, etc.
 
 > I like the ability to merge multiple .gw
 files (bases),
 > but I dislike the family
 centric design[1]. The internal places
 >
 and sources "tables/dictionnaries" are maybe
 oriented "geneanet",
 > like the
 missing repository support and the media handling, but
 > as the local version is available and as
 the geneweb file
 > format is open (not
 documented ...still open document ?),
 >
 maybe to still (try to) support this file format is a good
 feature.
 >
 > [1] https://gramps-project.org/bugs/view.php?id=2488
 I agree, but it's quite common on the web.
 I have a few cousins who are
 not really
 cousins because their biological father is not the
 registered
 one, and GeneaNet only shows the
 biological one. Apparently, when
 importing
 from GEDCOM, GeneWeb shows the other one away, and maybe
 these
 cousins are a cause for gw import
 going wrong. Sites like Ancestry, Geni
 and
 My Heritage have similar restrictions, and on Ancestry and
 My
 Heritage, my GEDCOM imports simply fail.
 Ancestry gives me a useless
 error message,
 like GeneaNet does with gw import, and on My Heritage the
 
 import simply quits, meaning that they
 promise to send an email when the
 import is
 done, and that email never comes.
 
 If we can afford it, I think that supporting
 the gw format is a good
 idea indeed,
 because GeneaNet is big on the European continent. We may
 
 need to use plug-ins though, because
 GeneWeb software downloads are
 still
 version 6, mentioning 7 as alpha, and GeneaNet already runs
 on 7!
 
 I just downloaded my
 tree from GeneaNet to run a simple test, and plan
 to report about that later.
 
 regards,
 
 Enno
 
 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: GeneWeb [was Before gramps-4.1.3]

enno
In reply to this post by jerome
Jerome, Eric,

To keep much needed focus, I'm skipping your other message for now.
> Looks like gramps already deprecates .gw file format support
> (geneweb 4?) on import, and might try to support gwplus (geneweb 5? 6, 7?)?
> It is right that the new geneweb file format sounds much better.
It looks like that, yes. I applied your attached patch to 4.1, and it
got pretty far reading the gw that I downloaded from GeneaNet, which is
gwplus indeed. So it's probably safe to say that 7 = gwplus.

Pretty far means that I found a weird error importing baptism dates, for
which I may have a patch, but after that, I found another more nasty
error importing citations, so I stopped.

> Thanks to logging and recent parser review on import geneweb module.
> Debug is simplier when we get these complete LOG infos. :-)
Indeed.
> event records : personal events, family events, etc ...
> outside a top level like family (or person), is close to gramps' way.
>
> It takes me few minutes for generating a quick draft (proof) for
> the import. We could easily translate these events with gramps types
> and the parsing should be more consistent.
> I did not look at way, which is linking these events to person or family
> (could be really the name as key ???), but it seems we
> could provide a better geneweb support, at least on import.
Yes, import is the easiest in so far that with debug on it's pretty easy
to see what's going on, and wrong. Export relies on the quality of
gwplus documentation, errors generated by geneweb, or on-line, and the
latter are not shown on GeneaNet, so for that we're in the dark.

But for me, knowing that I can spend time on a lot of other things, like
discussions on FHSIO about source and citations, and places, which are a
mess in my tree, it's also important to think in a strategic way, like
whether we want to support GeneWeb for GeneaNet or for users that use it
for their own site. GeneaNet works well with Gramps GEDCOMs, even with
multiple parent families, and I don't make changes on-line, so in
reality, I don't import GEDCOM or GWplus from GeneaNet at all. And
knowing that we don't have a working GEDCOM or GW import merge, I doubt
that many will use that import often.

GeneWeb import then is only important for users that transfer from
GeneWeb to Gramps, either because they ran their own GeneWeb, or they
want to move from working on-line on GeneaNet to Gramps. In that case,
export is also nice if it gives more fields than GEDCOM, like the
witnesses, and and maybe a couple of other things. Export to run your
own site on GeneWeb looks quite weird to me, because we assume that
Gramps is so good, that users prefer Gramps connect for that.

Going back and forth between GeneWeb and Gramps looks quite silly to me,
because it causes more data loss than most other GEDCOM based programs
available today.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Before gramps-4.1.3

jerome
In reply to this post by enno
We did not test geneweb import with python3, where it seems to need the same type of code...
I suppose it crashes on current gramps41 branch under python3 too!

I am using a quick workaround under my local 'master/trunk' version:

-            self.f = open(file, "rU")
+           self.f = open(file, "rUb")

I would like to provide at least a basic and stable support on import for the
gwplus file format. To include gwplus support as a new feature on master
should be easy.

self.gwplus = False
    ...
    if fields[0] == "gwplus":
        self.gwplus = True
    ...
    if gwplus:
        self.encoding = 'utf-8'
        ...

So, this will be in addition to .gw support.


Jérôme




--------------------------------------------
En date de : Sam 2.5.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] Before gramps-4.1.3
 À: "jerome" <[hidden email]>, [hidden email]
 Date: Samedi 2 mai 2015, 17h11
 
 Jerome,
 
 > I hope so; only supposing that GeneWeb is still able to
 convert iso-8859!
 > https://github.com/geneanet/geneweb/blob/master/src/mutil.ml
 > https://github.com/geneanet/geneweb/blob/master/gwb2ged/gwb2ged.ml
 > etc...
 
 > True, maybe a .gw file format[1] review for gramps
 4.2.
 > But that's something else.
 
 > [1] https://gramps-project.org/bugs/view.php?id=3094
 
 I guess it won't be easy without more details from the
 GeneWeb side, like more detailed error information.
 
 I uploaded a .gw file to GeneaNet today, exported from
 3.4.9, and got a message that there's something wrong with
 my GEDCOM file! No further details other than that the file
 is invalid, i.e. incorrect extension or damaged.
 
 I haven't tested with a GeneWeb server on Linux or Windows,
 but with this lack of information, I don't see many ways to
 improve in Gramps.
 
 Do you have any contacts on GeneaNet? It's a popular site in
 Europe, and uploads in GeneWeb format may be better than
 GEDCOM, if they work.
 
 regards,
 
 Enno
 
 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Before gramps-4.1.3

enno
Hi Jerome,
> We did not test geneweb import with python3, where it seems to need the same type of code...
> I suppose it crashes on current gramps41 branch under python3 too!
It does indeed, but not on the file open that you mentioned. There's a
list index error

>  File
> "/home/enno/gramps-enno/gramps/plugins/importer/importgeneweb.py",
> line 647, in parse_person
>     LOG.debug("Baptize at: %s" % fields[idx])
> IndexError: list index out of range
caused by this piece of code
>             elif field[0] == '!':
>                 LOG.debug("Baptize at: %s" % fields[idx])
>                 bapt_date = self.parse_date(self.decode(fields[idx][1:]))
>                 idx += 1
In the GW file, I see that the baptism date is in field[1:-1], not
fields[idx], and because of that, I guess that the increment of idx is
wrong too.

> I am using a quick workaround under my local 'master/trunk' version:
>
> -            self.f = open(file, "rU")
> +           self.f = open(file, "rUb")
I don't understand that, because except for the above, the import seems
to work.

> I would like to provide at least a basic and stable support on import for the
> gwplus file format. To include gwplus support as a new feature on master
> should be easy.
>
> self.gwplus = False
>      ...
>      if fields[0] == "gwplus":
>          self.gwplus = True
>      ...
>      if gwplus:
>          self.encoding = 'utf-8'
>          ...
The encoding is in the GW file itself:
> encoding: utf-8
> gwplus
So I think it should not be implied by the gwplus format.

Anyway, to my regret, my priorities are still elsewhere. I may have time
to do some tests with my gwplus download, but that's about it, sorry.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Before gramps-4.1.3

jerome
Hi Enno,


Whatever change on my test file, it raises the same error after n lines!

gramps/plugins/importer/importgeneweb.py", line 109, in get_next_line
    line = self.f.readline()
  File "/usr/lib/python3.4/codecs.py", line 313, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 574: invalid continuation byte

Parsing the file passes if I read the file with the workaroud, but it also means that
fields are now parsed in a "byte mode", isn't it?

https://wiki.python.org/moin/Python3UnicodeDecodeError

Anyway, I just want to parse and import most data.
So, the UnicodeDecodeError could be fixed later.

Once parser improved for supporting this "plus"
version of the gw format, we could ask for more tests
on master (Eric, Bastien, etc...) have a geneanet
account.

For now, my problem is rather to have something a little bit stable
on gwplus specification!

https://github.com/geneanet/geneweb/commit/4029a86ad0eb93c91f7d2ffa748071494b0172bd

About the gedcom and the gwplus file format,
we can see a map too on ged2gwb/ged2gwb.ml
file. (above link)

Note, we are in a AND logic, than a OR one!

For both specification (gw & gwplus) and file
formats (.gw & .ged). Supporting mutilple
file formats does not mean that one should
be the exchange format.

And people with a geneanet account are making
backup with the geneweb file format.

There is still a lot of bug reports (set of reports)
around the geneweb file format on bug tracker.
A review on .gw import for gramps 4.2 should be
not very experimental (export will be something else...)


regards,
Jérôme


--------------------------------------------
En date de : Lun 11.5.15, Enno Borgsteede <[hidden email]> a écrit :

 Objet: Re: [Gramps-devel] Before gramps-4.1.3
 À: "jerome" <[hidden email]>, [hidden email]
 Date: Lundi 11 mai 2015, 23h00
 
 Hi Jerome,
 > We did not test geneweb import with
 python3, where it seems to need the same type of code...
 > I suppose it crashes on current gramps41
 branch under python3 too!
 It does indeed,
 but not on the file open that you mentioned. There's a
 
 list index error
 
 >  File
 >
 "/home/enno/gramps-enno/gramps/plugins/importer/importgeneweb.py",
 
 > line 647, in parse_person
 >     LOG.debug("Baptize
 at: %s" % fields[idx])
 > IndexError:
 list index out of range
 caused by this piece
 of code
 >         
    elif field[0] == '!':
 >             
    LOG.debug("Baptize at: %s" %
 fields[idx])
 >             
    bapt_date =
 self.parse_date(self.decode(fields[idx][1:]))
 >                 idx +=
 1
 In the GW file, I see that the baptism
 date is in field[1:-1], not
 fields[idx],
 and because of that, I guess that the increment of idx is
 
 wrong too.
 
 > I am using a quick workaround under my
 local 'master/trunk' version:
 >
 > -           
 self.f = open(file, "rU")
 > + 
          self.f = open(file,
 "rUb")
 I don't understand
 that, because except for the above, the import seems
 to work.
 > I would like to
 provide at least a basic and stable support on import for
 the
 > gwplus file format. To include
 gwplus support as a new feature on master
 > should be easy.
 >
 > self.gwplus = False
 > 
     ...
 >      if fields[0] ==
 "gwplus":
 >         
 self.gwplus = True
 >      ...
 >      if gwplus:
 > 
         self.encoding = 'utf-8'
 >          ...
 The
 encoding is in the GW file itself:
 >
 encoding: utf-8
 > gwplus
 So I think it should not be implied by the
 gwplus format.
 
 Anyway, to
 my regret, my priorities are still elsewhere. I may have
 time
 to do some tests with my gwplus
 download, but that's about it, sorry.
 
 regards,
 
 Enno
 
 

------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

GeneWeb import

enno
Hi Jerome,

> Whatever change on my test file, it raises the same error after n lines!
>
> gramps/plugins/importer/importgeneweb.py", line 109, in get_next_line
>      line = self.f.readline()
>    File "/usr/lib/python3.4/codecs.py", line 313, in decode
>      (result, consumed) = self._buffer_decode(data, self.errors, final)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd0 in position 574: invalid continuation byte
>
> Parsing the file passes if I read the file with the workaroud, but it also means that
> fields are now parsed in a "byte mode", isn't it?
Yes, that's right. My test file is a base.gw coming from GeneaNet, and
that one's UTF-8, so I don't need your patch. And thinking about the
GEDCOM import, I guess that the easiest for you is to do something
similar to what that importer does, meaning that you close and open the
file again if the initial open does not match with the encoding. For the
GeneWeb importer that means that you can continue reading when it's
UTF-8, but need to re-open as binary if it's ISO-something (bytes).

> Once parser improved for supporting this "plus"
> version of the gw format, we could ask for more tests
> on master (Eric, Bastien, etc...) have a geneanet
> account.
That's what I'm doing with my censored full tree on GeneaNet. With my
own patch for the baptism date, it seems to run until the end, and then
throws a DB exception:

> 2015-05-12 14:16:38.664: WARNING: importgeneweb.py: line 516:
> parse_marriage(): Unknown field: '?' in line 246731!
> 2015-05-12 14:16:38.931: WARNING: importgeneweb.py: line 516:
> parse_marriage(): Unknown field: 'Kirkham,_Yorkshire,_England' in line
> 247138!
> 2015-05-12 14:16:48.690: ERROR: grampsapp.py: line 107: Unhandled
> exception
> TypeError: DB associate callback should return
> DB_DONOTINDEX/bytes/list of bytes.
This exception comes back when I kill Gramps with Ctrl-\ and then open
the same test tree, so it looks like a permanent mess-up. This is in
master, and I have not tested this with gramps41 and Python 2. I did use
your 8548_2 patch though. An earlier session with debug shows that the
GW file was fully parsed.

regards,

Enno


------------------------------------------------------------------------------
One dashboard for servers and applications across Physical-Virtual-Cloud
Widest out-of-the-box monitoring support with 50+ applications
Performance metrics, stats and reports that give you Actionable Insights
Deep dive visibility with transaction tracing using APM Insight.
http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
12