Filters and regular expressions

classic Classic list List threaded Threaded
48 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Filters and regular expressions

Doug
I think there was some discussion a while ago about making
all filters capable of using regular expressions - what are
the current state and intentions?

The question raised itself recently when I wanted to filter
for things like Birth Events lacking a Date, or Birth Events
lacking a Place, by putting "^$" in the appropriate field.
The filter fails to select any Events.
The filter editor has no "Use regular expression" option and
the filter sidebar "Use regular expression" has no effect.

Doug


P.S. gramps 4.0.2 (r22816)

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Nick Hall-6
On 08/08/13 12:22, doug wrote:
> I think there was some discussion a while ago about making
> all filters capable of using regular expressions - what are
> the current state and intentions?

I have a patch for v4.0 that adds regular expression functionality to
some rules.  It enables the functionality available in the base class
and combines a few rules, where there is a separate rule for regular
expressions.

I'll check to see how far I got with it.

Nick.


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Nick Hall-6
Devs,

I have now added regular expression matching to all applicable rules in
gramps40 and trunk.  In doing so I have renamed "containing substring"
and "matching regular expression" to "containing text".

The regular expression checkbox contains a tooltip which briefly
explains the functionality.

The following rules optionally accept regular expressions:

Person

People with the <citation>
People with the <birth data>
People with the <death data>
People with the family <event>
People with the personal <event>
People having notes containing <text>
People with Id containing <text>
People with the family <attribute>
People with the personal <attribute>

Family

Families having child with Id containing <text>
Families with the <citation>
Families with the <event>
Families having father with Id containing <text>
Families having notes containing <text>
Families with Id containing <text>
Families with the family <attribute>
Families having mother with Id containing <text>

Event

Events with the <citation>
Events having notes containing <text>
Events with Id containing <text>
Events with <data>
Events with the attribute <attribute>

Place

Place with the <citation>
Places having notes containing <text>
Places matching parameters
Places with Id containing <text>

Source

Sources having notes containing <text>
Sources title containing <text>
Sources with Id containing <text>
Sources with repository reference containing <text> in "Call Number"

Citation

Citations with Volume/Page containing <text>
Citations having notes containing <text>
Citations matching parameters
Citations with Id containing <text>
Sources matching parameters

Repository

Repositories having notes containing <text>
Repositories with Id containing <text>
Repositories with name containing <text>

Media

Media with the <citation>
Media objects with Id containing <text>
Media objects having notes containing <text>
Media objects with the attribute <attribute>

Note

Notes containing <text>
Notes matching parameters
Notes with <Id> matching regular expression

There are five rules which previously accepted regular expressions. I
have left these unchanged:

People with records containing <substring>
People with the <name>
Families with child with the <Name>
Families with father with the <Name>
Families with mother with the <Name>

Please give the updated rules some testing and let me know what you think.

Regards,

Nick.


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Doug
On 09/08/13 00:08, Nick Hall wrote:

> Devs,
>
> I have now added regular expression matching to all applicable rules in
> gramps40 and trunk.  In doing so I have renamed "containing substring"
> and "matching regular expression" to "containing text".
>
> The regular expression checkbox contains a tooltip which briefly
> explains the functionality.
>
> The following rules optionally accept regular expressions:
>
> Person
>
> People with the <citation>
> People with the <birth data>
> People with the <death data>
> People with the family <event>
> People with the personal <event>
> People having notes containing <text>
> People with Id containing <text>
> People with the family <attribute>
> People with the personal <attribute>
>
> Family
>
> Families having child with Id containing <text>
> Families with the <citation>
> Families with the <event>
> Families having father with Id containing <text>
> Families having notes containing <text>
> Families with Id containing <text>
> Families with the family <attribute>
> Families having mother with Id containing <text>
>
> Event
>
> Events with the <citation>
> Events having notes containing <text>
> Events with Id containing <text>
> Events with <data>
> Events with the attribute <attribute>
>
> Place
>
> Place with the <citation>
> Places having notes containing <text>
> Places matching parameters
> Places with Id containing <text>
>
> Source
>
> Sources having notes containing <text>
> Sources title containing <text>
> Sources with Id containing <text>
> Sources with repository reference containing <text> in "Call Number"
>
> Citation
>
> Citations with Volume/Page containing <text>
> Citations having notes containing <text>
> Citations matching parameters
> Citations with Id containing <text>
> Sources matching parameters
>
> Repository
>
> Repositories having notes containing <text>
> Repositories with Id containing <text>
> Repositories with name containing <text>
>
> Media
>
> Media with the <citation>
> Media objects with Id containing <text>
> Media objects having notes containing <text>
> Media objects with the attribute <attribute>
>
> Note
>
> Notes containing <text>
> Notes matching parameters
> Notes with <Id> matching regular expression
>
> There are five rules which previously accepted regular expressions. I
> have left these unchanged:
>
> People with records containing <substring>
> People with the <name>
> Families with child with the <Name>
> Families with father with the <Name>
> Families with mother with the <Name>
>
> Please give the updated rules some testing and let me know what you think.
>
> Regards,
>
> Nick.


Terrific!
I'll report back when I've had a chance to try things out.

Doug


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Doug
On 09/08/13 10:24, doug wrote:

> On 09/08/13 00:08, Nick Hall wrote:
>> Devs,
>>
>> I have now added regular expression matching to all applicable rules in
>> gramps40 and trunk.  In doing so I have renamed "containing substring"
>> and "matching regular expression" to "containing text".
>>
>> The regular expression checkbox contains a tooltip which briefly
>> explains the functionality.
>>
>> The following rules optionally accept regular expressions:
>>
>> Person
>>
>> People with the <citation>
>> People with the <birth data>
>> People with the <death data>
>> People with the family <event>
>> People with the personal <event>
>> People having notes containing <text>
>> People with Id containing <text>
>> People with the family <attribute>
>> People with the personal <attribute>
>>
>> Family
>>
>> Families having child with Id containing <text>
>> Families with the <citation>
>> Families with the <event>
>> Families having father with Id containing <text>
>> Families having notes containing <text>
>> Families with Id containing <text>
>> Families with the family <attribute>
>> Families having mother with Id containing <text>
>>
>> Event
>>
>> Events with the <citation>
>> Events having notes containing <text>
>> Events with Id containing <text>
>> Events with <data>
>> Events with the attribute <attribute>
>>
>> Place
>>
>> Place with the <citation>
>> Places having notes containing <text>
>> Places matching parameters
>> Places with Id containing <text>
>>
>> Source
>>
>> Sources having notes containing <text>
>> Sources title containing <text>
>> Sources with Id containing <text>
>> Sources with repository reference containing <text> in "Call Number"
>>
>> Citation
>>
>> Citations with Volume/Page containing <text>
>> Citations having notes containing <text>
>> Citations matching parameters
>> Citations with Id containing <text>
>> Sources matching parameters
>>
>> Repository
>>
>> Repositories having notes containing <text>
>> Repositories with Id containing <text>
>> Repositories with name containing <text>
>>
>> Media
>>
>> Media with the <citation>
>> Media objects with Id containing <text>
>> Media objects having notes containing <text>
>> Media objects with the attribute <attribute>
>>
>> Note
>>
>> Notes containing <text>
>> Notes matching parameters
>> Notes with <Id> matching regular expression
>>
>> There are five rules which previously accepted regular expressions. I
>> have left these unchanged:
>>
>> People with records containing <substring>
>> People with the <name>
>> Families with child with the <Name>
>> Families with father with the <Name>
>> Families with mother with the <Name>
>>
>> Please give the updated rules some testing and let me know what you think.
>>
>> Regards,
>>
>> Nick.
>
>
> Terrific!
> I'll report back when I've had a chance to try things out.
>
> Doug




First report:

Place filters work fine in 3.4.6 (python 2.7) except that
Place with the <citation>  Date:="^$" doesn't select missing
dates.


only one Event filter tested so far:
Events with the <citation> Citation Volume/Page:="^$" to
select missing Vol/Pages crashes gramps.
Error Details:
===================

1306385: ERROR: gramps.py: line 107: Unhandled exception
Traceback (most recent call last):
   File
"/usr/share/gramps34/src/Filters/SideBar/_SidebarFilter.py",
line 109, in clicked
     self.clicked_func()
   File
"/usr/share/gramps34/src/plugins/gramplet/Filter.py", line
60, in __filter_clicked
     self.gui.view.build_tree()
   File "/usr/share/gramps34/src/gui/views/listview.py",
line 286, in build_tree
     self.model.rebuild_data()
   File
"/usr/share/gramps34/src/gui/views/treemodels/flatbasemodel.py",
line 559, in _rebuild_filter
     dlist = self.search.apply(self.db, allkeys, tupleind=1)
   File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
line 233, in apply
     res = m(db, id_list, cb_progress, tupleind)
   File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
line 166, in check_and
     val = all(rule.apply(db, person) for rule in flist if
person)
   File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
line 166, in <genexpr>
     val = all(rule.apply(db, person) for rule in flist if
person)
   File
"/usr/share/gramps34/src/Filters/Rules/_MatchesFilterBase.py",
line 79, in apply
     return filt.check(db, obj.handle)
   File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
line 209, in check
     return self.get_check_func()(db, [handle])
   File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
line 166, in check_and
     val = all(rule.apply(db, person) for rule in flist if
person)
   File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
line 166, in <genexpr>
     val = all(rule.apply(db, person) for rule in flist if
person)
   File
"/usr/share/gramps34/src/Filters/Rules/Event/_HasCitation.py",
line 59, in apply
     if HasCitationBase.apply(self, dbase, citation):
   File
"/usr/share/gramps34/src/Filters/Rules/_HasCitationBase.py",
line 67, in apply
     for citation_handle in object.get_citation_list():
AttributeError: 'Citation' object has no attribute
'get_citation_list'


System Information:
===================

Python version: 2.7.3rc2 (default, Apr 22 2012, 22:30:17)
[GCC 4.6.3]
BSDDB version: 5.1.2 (5, 1, 29)
Gramps version: 3.4.6-0.SVN22823
LANG: en_GB.UTF-8
OS: Linux
Distribution: 3.2.0-4-amd64

GTK version    : (2, 24, 10)
pygtk version  : (2, 24, 0)
gobject version: (2, 28, 6)
cairo version  : (1, 8, 8)



Doug


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Doug
On 09/08/13 12:39, doug wrote:

> On 09/08/13 10:24, doug wrote:
>> On 09/08/13 00:08, Nick Hall wrote:
>>> Devs,
>>>
>>> I have now added regular expression matching to all
>>> applicable rules in
>>> gramps40 and trunk.  In doing so I have renamed
>>> "containing substring"
>>> and "matching regular expression" to "containing text".
>>>
>>> The regular expression checkbox contains a tooltip which
>>> briefly
>>> explains the functionality.
>>>
>>> The following rules optionally accept regular expressions:
>>>
>>> Person
>>>
>>> People with the <citation>
>>> People with the <birth data>
>>> People with the <death data>
>>> People with the family <event>
>>> People with the personal <event>
>>> People having notes containing <text>
>>> People with Id containing <text>
>>> People with the family <attribute>
>>> People with the personal <attribute>
>>>
>>> Family
>>>
>>> Families having child with Id containing <text>
>>> Families with the <citation>
>>> Families with the <event>
>>> Families having father with Id containing <text>
>>> Families having notes containing <text>
>>> Families with Id containing <text>
>>> Families with the family <attribute>
>>> Families having mother with Id containing <text>
>>>
>>> Event
>>>
>>> Events with the <citation>
>>> Events having notes containing <text>
>>> Events with Id containing <text>
>>> Events with <data>
>>> Events with the attribute <attribute>
>>>
>>> Place
>>>
>>> Place with the <citation>
>>> Places having notes containing <text>
>>> Places matching parameters
>>> Places with Id containing <text>
>>>
>>> Source
>>>
>>> Sources having notes containing <text>
>>> Sources title containing <text>
>>> Sources with Id containing <text>
>>> Sources with repository reference containing <text> in
>>> "Call Number"
>>>
>>> Citation
>>>
>>> Citations with Volume/Page containing <text>
>>> Citations having notes containing <text>
>>> Citations matching parameters
>>> Citations with Id containing <text>
>>> Sources matching parameters
>>>
>>> Repository
>>>
>>> Repositories having notes containing <text>
>>> Repositories with Id containing <text>
>>> Repositories with name containing <text>
>>>
>>> Media
>>>
>>> Media with the <citation>
>>> Media objects with Id containing <text>
>>> Media objects having notes containing <text>
>>> Media objects with the attribute <attribute>
>>>
>>> Note
>>>
>>> Notes containing <text>
>>> Notes matching parameters
>>> Notes with <Id> matching regular expression
>>>
>>> There are five rules which previously accepted regular
>>> expressions. I
>>> have left these unchanged:
>>>
>>> People with records containing <substring>
>>> People with the <name>
>>> Families with child with the <Name>
>>> Families with father with the <Name>
>>> Families with mother with the <Name>
>>>
>>> Please give the updated rules some testing and let me
>>> know what you think.
>>>
>>> Regards,
>>>
>>> Nick.
>>
>>
>> Terrific!
>> I'll report back when I've had a chance to try things out.
>>
>> Doug
>
>
>
>
> First report:
>
> Place filters work fine in 3.4.6 (python 2.7) except that
> Place with the <citation>  Date:="^$" doesn't select missing
> dates.
>
>
> only one Event filter tested so far:
> Events with the <citation> Citation Volume/Page:="^$" to
> select missing Vol/Pages crashes gramps.
> Error Details:
> ===================
>
> 1306385: ERROR: gramps.py: line 107: Unhandled exception
> Traceback (most recent call last):
>    File
> "/usr/share/gramps34/src/Filters/SideBar/_SidebarFilter.py",
> line 109, in clicked
>      self.clicked_func()
>    File
> "/usr/share/gramps34/src/plugins/gramplet/Filter.py", line
> 60, in __filter_clicked
>      self.gui.view.build_tree()
>    File "/usr/share/gramps34/src/gui/views/listview.py",
> line 286, in build_tree
>      self.model.rebuild_data()
>    File
> "/usr/share/gramps34/src/gui/views/treemodels/flatbasemodel.py",
> line 559, in _rebuild_filter
>      dlist = self.search.apply(self.db, allkeys, tupleind=1)
>    File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
> line 233, in apply
>      res = m(db, id_list, cb_progress, tupleind)
>    File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
> line 166, in check_and
>      val = all(rule.apply(db, person) for rule in flist if
> person)
>    File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
> line 166, in <genexpr>
>      val = all(rule.apply(db, person) for rule in flist if
> person)
>    File
> "/usr/share/gramps34/src/Filters/Rules/_MatchesFilterBase.py",
> line 79, in apply
>      return filt.check(db, obj.handle)
>    File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
> line 209, in check
>      return self.get_check_func()(db, [handle])
>    File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
> line 166, in check_and
>      val = all(rule.apply(db, person) for rule in flist if
> person)
>    File "/usr/share/gramps34/src/Filters/_GenericFilter.py",
> line 166, in <genexpr>
>      val = all(rule.apply(db, person) for rule in flist if
> person)
>    File
> "/usr/share/gramps34/src/Filters/Rules/Event/_HasCitation.py",
> line 59, in apply
>      if HasCitationBase.apply(self, dbase, citation):
>    File
> "/usr/share/gramps34/src/Filters/Rules/_HasCitationBase.py",
> line 67, in apply
>      for citation_handle in object.get_citation_list():
> AttributeError: 'Citation' object has no attribute
> 'get_citation_list'
>
>
> System Information:
> ===================
>
> Python version: 2.7.3rc2 (default, Apr 22 2012, 22:30:17)
> [GCC 4.6.3]
> BSDDB version: 5.1.2 (5, 1, 29)
> Gramps version: 3.4.6-0.SVN22823
> LANG: en_GB.UTF-8
> OS: Linux
> Distribution: 3.2.0-4-amd64
>
> GTK version    : (2, 24, 10)
> pygtk version  : (2, 24, 0)
> gobject version: (2, 28, 6)
> cairo version  : (1, 8, 8)
>
>
>
> Doug
>
P.S. similar crash and error with
  Events with the <citation> Citation Date:="^$"

Doug

------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Doug
On 09/08/13 12:47, doug wrote:

> On 09/08/13 12:39, doug wrote:
>> On 09/08/13 10:24, doug wrote:
>>> On 09/08/13 00:08, Nick Hall wrote:
>>>> Devs,
>>>>
>>>> I have now added regular expression matching to all
>>>> applicable rules in
>>>> gramps40 and trunk.  In doing so I have renamed
>>>> "containing substring"
>>>> and "matching regular expression" to "containing text".
>>>>
>>>> The regular expression checkbox contains a tooltip which
>>>> briefly
>>>> explains the functionality.
>>>>
>>>> The following rules optionally accept regular expressions:
>>>>
>>>> Person
>>>>
>>>> People with the <citation>
>>>> People with the <birth data>
>>>> People with the <death data>
>>>> People with the family <event>
>>>> People with the personal <event>
>>>> People having notes containing <text>
>>>> People with Id containing <text>
>>>> People with the family <attribute>
>>>> People with the personal <attribute>
>>>>
>>>> Family
>>>>
>>>> Families having child with Id containing <text>
>>>> Families with the <citation>
>>>> Families with the <event>
>>>> Families having father with Id containing <text>
>>>> Families having notes containing <text>
>>>> Families with Id containing <text>
>>>> Families with the family <attribute>
>>>> Families having mother with Id containing <text>
>>>>
>>>> Event
>>>>
>>>> Events with the <citation>
>>>> Events having notes containing <text>
>>>> Events with Id containing <text>
>>>> Events with <data>
>>>> Events with the attribute <attribute>
>>>>
>>>> Place
>>>>
>>>> Place with the <citation>
>>>> Places having notes containing <text>
>>>> Places matching parameters
>>>> Places with Id containing <text>
>>>>
>>>> Source
>>>>
>>>> Sources having notes containing <text>
>>>> Sources title containing <text>
>>>> Sources with Id containing <text>
>>>> Sources with repository reference containing <text> in
>>>> "Call Number"
>>>>
>>>> Citation
>>>>
>>>> Citations with Volume/Page containing <text>
>>>> Citations having notes containing <text>
>>>> Citations matching parameters
>>>> Citations with Id containing <text>
>>>> Sources matching parameters
>>>>
>>>> Repository
>>>>
>>>> Repositories having notes containing <text>
>>>> Repositories with Id containing <text>
>>>> Repositories with name containing <text>
>>>>
>>>> Media
>>>>
>>>> Media with the <citation>
>>>> Media objects with Id containing <text>
>>>> Media objects having notes containing <text>
>>>> Media objects with the attribute <attribute>
>>>>
>>>> Note
>>>>
>>>> Notes containing <text>
>>>> Notes matching parameters
>>>> Notes with <Id> matching regular expression
>>>>
>>>> There are five rules which previously accepted regular
>>>> expressions. I
>>>> have left these unchanged:
>>>>
>>>> People with records containing <substring>
>>>> People with the <name>
>>>> Families with child with the <Name>
>>>> Families with father with the <Name>
>>>> Families with mother with the <Name>
>>>>
>>>> Please give the updated rules some testing and let me
>>>> know what you think.
>>>>
>>>> Regards,
>>>>
>>>> Nick.
>>>
>>>
>>> Terrific!
>>> I'll report back when I've had a chance to try things out.
>>>
>>> Doug
>>
>>
>>
>>
>> First report:
>>
<snip>


Second report:

Events with <data> Date, Place, Description:="^$" doesn't
select missing; in fact regular expression doesn't seem to
work ( e.g.\d+ for dates)
Events with the attribute <attribute>  doesn't select with
or without regex

Sources title containing <text> is absent; Sources title
containing <substring> doesn't recognise regex

Sources with repository reference containing <text> in "Call
Number" is absent; Sources with repository reference
containing <substring> in "Call Number" doesn't recognise regex

Doug



------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Nick Hall-6
On 09/08/13 13:24, doug wrote:

> Second report:
>
> Events with <data> Date, Place, Description:="^$" doesn't select
> missing; in fact regular expression doesn't seem to work ( e.g.\d+ for
> dates)
> Events with the attribute <attribute>  doesn't select with or without
> regex
>
> Sources title containing <text> is absent; Sources title containing
> <substring> doesn't recognise regex
>
> Sources with repository reference containing <text> in "Call Number"
> is absent; Sources with repository reference containing <substring> in
> "Call Number" doesn't recognise regex
>

Doug,

I only made the changes in gramps40 and trunk, not gramps34.

Dates use special matching that takes into account approximate dates and
date ranges.  You must enter a date that can be parsed by Gramps, rather
than a regular expression.

Gramps types are normally selected from a drop-down list.  I haven't
implemented regular expression matching for these fields, although it
would be possible to do so.

When we are happy with the changes in gramps40, I could backport them to
gramps34.

Nick.


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Nick Hall-6
In reply to this post by Doug
Doug,

My changes were in gramps40 and trunk only.

I suggest you raise a bug for this, it will be fixed anyway if we backport my regular expression changes to gramps34.

Nick.


On 09/08/13 12:39, doug wrote:
First report:

Place filters work fine in 3.4.6 (python 2.7) except that
Place with the <citation>  Date:="^$" doesn't select missing dates.


only one Event filter tested so far:
Events with the <citation> Citation Volume/Page:="^$" to select missing Vol/Pages crashes gramps.
Error Details:
===================

1306385: ERROR: gramps.py: line 107: Unhandled exception
Traceback (most recent call last):
  File "/usr/share/gramps34/src/Filters/SideBar/_SidebarFilter.py", line 109, in clicked
    self.clicked_func()
  File "/usr/share/gramps34/src/plugins/gramplet/Filter.py", line 60, in __filter_clicked
    self.gui.view.build_tree()
  File "/usr/share/gramps34/src/gui/views/listview.py", line 286, in build_tree
    self.model.rebuild_data()
  File "/usr/share/gramps34/src/gui/views/treemodels/flatbasemodel.py", line 559, in _rebuild_filter
    dlist = self.search.apply(self.db, allkeys, tupleind=1)
  File "/usr/share/gramps34/src/Filters/_GenericFilter.py", line 233, in apply
    res = m(db, id_list, cb_progress, tupleind)
  File "/usr/share/gramps34/src/Filters/_GenericFilter.py", line 166, in check_and
    val = all(rule.apply(db, person) for rule in flist if person)
  File "/usr/share/gramps34/src/Filters/_GenericFilter.py", line 166, in <genexpr>
    val = all(rule.apply(db, person) for rule in flist if person)
  File "/usr/share/gramps34/src/Filters/Rules/_MatchesFilterBase.py", line 79, in apply
    return filt.check(db, obj.handle)
  File "/usr/share/gramps34/src/Filters/_GenericFilter.py", line 209, in check
    return self.get_check_func()(db, [handle])
  File "/usr/share/gramps34/src/Filters/_GenericFilter.py", line 166, in check_and
    val = all(rule.apply(db, person) for rule in flist if person)
  File "/usr/share/gramps34/src/Filters/_GenericFilter.py", line 166, in <genexpr>
    val = all(rule.apply(db, person) for rule in flist if person)
  File "/usr/share/gramps34/src/Filters/Rules/Event/_HasCitation.py", line 59, in apply
    if HasCitationBase.apply(self, dbase, citation):
  File "/usr/share/gramps34/src/Filters/Rules/_HasCitationBase.py", line 67, in apply
    for citation_handle in object.get_citation_list():
AttributeError: 'Citation' object has no attribute 'get_citation_list'


System Information:
===================

Python version: 2.7.3rc2 (default, Apr 22 2012, 22:30:17) [GCC 4.6.3]
BSDDB version: 5.1.2 (5, 1, 29)
Gramps version: 3.4.6-0.SVN22823
LANG: en_GB.UTF-8
OS: Linux
Distribution: 3.2.0-4-amd64

GTK version    : (2, 24, 10)
pygtk version  : (2, 24, 0)
gobject version: (2, 28, 6)
cairo version  : (1, 8, 8)


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Doug
In reply to this post by Nick Hall-6
On 09/08/13 14:23, Nick Hall wrote:

> On 09/08/13 13:24, doug wrote:
>> Second report:
>>
>> Events with <data> Date, Place, Description:="^$" doesn't
>> select missing; in fact regular expression doesn't seem to
>> work ( e.g.\d+ for dates)
>> Events with the attribute <attribute>  doesn't select with
>> or without regex
>>
>> Sources title containing <text> is absent; Sources title
>> containing <substring> doesn't recognise regex
>>
>> Sources with repository reference containing <text> in
>> "Call Number" is absent; Sources with repository reference
>> containing <substring> in "Call Number" doesn't recognise
>> regex
>>
>
> Doug,
>
> I only made the changes in gramps40 and trunk, not gramps34.
>
> Dates use special matching that takes into account
> approximate dates and date ranges.  You must enter a date
> that can be parsed by Gramps, rather than a regular expression.
>
> Gramps types are normally selected from a drop-down list.  I
> haven't implemented regular expression matching for these
> fields, although it would be possible to do so.
>
> When we are happy with the changes in gramps40, I could
> backport them to gramps34.
>
> Nick.
>

OK.
I'll go back to gramps 40 and start again.

Doug


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Nick Hall-6
On 09/08/13 17:40, doug wrote:

> On 09/08/13 14:23, Nick Hall wrote:
>> On 09/08/13 13:24, doug wrote:
>>> Second report:
>>>
>>> Events with <data> Date, Place, Description:="^$" doesn't
>>> select missing; in fact regular expression doesn't seem to
>>> work ( e.g.\d+ for dates)
>>> Events with the attribute <attribute>  doesn't select with
>>> or without regex
>>>
>>> Sources title containing <text> is absent; Sources title
>>> containing <substring> doesn't recognise regex
>>>
>>> Sources with repository reference containing <text> in
>>> "Call Number" is absent; Sources with repository reference
>>> containing <substring> in "Call Number" doesn't recognise
>>> regex
>>>
>>
>> Doug,
>>
>> I only made the changes in gramps40 and trunk, not gramps34.
>>
>> Dates use special matching that takes into account
>> approximate dates and date ranges.  You must enter a date
>> that can be parsed by Gramps, rather than a regular expression.
>>
>> Gramps types are normally selected from a drop-down list.  I
>> haven't implemented regular expression matching for these
>> fields, although it would be possible to do so.
>>
>> When we are happy with the changes in gramps40, I could
>> backport them to gramps34.
>>
>> Nick.
>>
>
> OK.
> I'll go back to gramps 40 and start again.
>
> Doug
>
>
>
Doug,

I have now converted the five old rules to use the regular expression
functionality in the base class.

I have also backported the changes to gramps34.

Let me know if you have any problems.

Nick.


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Serge Noiraud-2
Hi,

Le 09/08/2013 23:40, Nick Hall a écrit :
> On 09/08/13 17:40, doug wrote:
>> On 09/08/13 14:23, Nick Hall wrote:
>>> On 09/08/13 13:24, doug wrote:
>>>> Second report:
...
> Doug,
>
> I have now converted the five old rules to use the regular expression
> functionality in the base class.
>
> I have also backported the changes to gramps34.
It doesn't work any more for me :
I get the following messages for all my filters:

WARNING: Too many arguments in filter « XXXXX » !
Trying to load with subset of arguments.

I needed to edit all the filters and put again "use regexp".
Now all filters are working again
>
> Let me know if you have any problems.
>
> Nick.


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Nick Hall-6
On 11/08/13 09:59, Serge Noiraud wrote:
> It doesn't work any more for me :
> I get the following messages for all my filters:
>
> WARNING: Too many arguments in filter « XXXXX » !
> Trying to load with subset of arguments.
>
> I needed to edit all the filters and put again "use regexp".
> Now all filters are working again

Serge,

Thanks for reporting this.

The warning occurred with the five upgraded rules that already used
regular expressions.

People with records containing <substring>
People with the <name>
Families with child with the <Name>
Families with father with the <Name>
Families with mother with the <Name>

I've added some code to automatically update these rules when they are
first loaded.  Let me know if this fixes your problem.

Nick.


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

ACProctor
I'm not a fan of text-based filters for this type of application. I know
through experience that they can be fragile and error-prone, especially if
misapplied to an idea that would be safer involving "objects" rather than
text.

Just to stir things up a little, can I refer you to a FHISO paper that does
seem to be relevant here: http://fhiso.org/files/cfp/cfps19.pdf

    Tony Proctor

----- Original Message -----
From: "Nick Hall" <[hidden email]>
To: "Serge Noiraud" <[hidden email]>
Cc: "Gramps-developers mailing list" <[hidden email]>
Sent: Sunday, August 11, 2013 3:16 PM
Subject: Re: [Gramps-devel] Filters and regular expressions


> On 11/08/13 09:59, Serge Noiraud wrote:
>> It doesn't work any more for me :
>> I get the following messages for all my filters:
>>
>> WARNING: Too many arguments in filter « XXXXX » !
>> Trying to load with subset of arguments.
>>
>> I needed to edit all the filters and put again "use regexp".
>> Now all filters are working again
>
> Serge,
>
> Thanks for reporting this.
>
> The warning occurred with the five upgraded rules that already used
> regular expressions.
>
> People with records containing <substring>
> People with the <name>
> Families with child with the <Name>
> Families with father with the <Name>
> Families with mother with the <Name>
>
> I've added some code to automatically update these rules when they are
> first loaded.  Let me know if this fixes your problem.
>
> Nick.
>
>
> ------------------------------------------------------------------------------
> Get 100% visibility into Java/.NET code with AppDynamics Lite!
> It's a free troubleshooting tool designed for production.
> Get down to code-level detail for bottlenecks, with <2% overhead.
> Download for free and get started troubleshooting in minutes.
> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
> _______________________________________________
> Gramps-devel mailing list
> [hidden email]
> https://lists.sourceforge.net/lists/listinfo/gramps-devel
>


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Serge Noiraud-2
Le 11/08/2013 16:22, Tony Proctor a écrit :
> I'm not a fan of text-based filters for this type of application. I know through experience that they can be fragile and error-prone, especially if misapplied to an idea that would be safer involving "objects" rather than text.
>
> Just to stir things up a little, can I refer you to a FHISO paper that does seem to be relevant here: http://fhiso.org/files/cfp/cfps19.pdf

Perhaps you are right,

But how do you search for search a string  ?

n(e|é|es|ey|oua|oai|o[iya](e|)|a[iy])r(r|)(o(n|t|d|s)|((e|)au(x|t|d|lt|)))

I use this regexp for all the possibilities I found for the name "noiraud" since 1515.

>
>    Tony Proctor
>
> ----- Original Message ----- From: "Nick Hall" <[hidden email]>
> To: "Serge Noiraud" <[hidden email]>
> Cc: "Gramps-developers mailing list" <[hidden email]>
> Sent: Sunday, August 11, 2013 3:16 PM
> Subject: Re: [Gramps-devel] Filters and regular expressions
>
>
>> On 11/08/13 09:59, Serge Noiraud wrote:
>>> It doesn't work any more for me :
>>> I get the following messages for all my filters:
>>>
>>> WARNING: Too many arguments in filter « XXXXX » !
>>> Trying to load with subset of arguments.
>>>
>>> I needed to edit all the filters and put again "use regexp".
>>> Now all filters are working again
>>
>> Serge,
>>
>> Thanks for reporting this.
>>
>> The warning occurred with the five upgraded rules that already used
>> regular expressions.
>>
>> People with records containing <substring>
>> People with the <name>
>> Families with child with the <Name>
>> Families with father with the <Name>
>> Families with mother with the <Name>
>>
>> I've added some code to automatically update these rules when they are
>> first loaded.  Let me know if this fixes your problem.
>>
>> Nick.
>>
>
>


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

ACProctor
With an object-based script, searching is much easier, and much more
readable and maintainable. It all depends what methods you define on your
objects.

I haven't looked at that FHISO paper for a while but I think it emphasises
that a method searching for, say, "John Smith" would automatically apply the
normal name-matching semantics for the associated software product. This
means it would treat the name case-blind, possibly accent-blind, and check
all the alternative names for each person, and handle abbreviations and
initials too. Hence, in some circumstances, a RegEx wouldn't be necessary
because the semantics of what you're trying to achieve are provided by a
supported method.

I'm sure a lot of effort went into your RegEx expression but I'd hate to be
the person who ended up maintaining it  ;-)

    Tony Proctor


----- Original Message -----
From: "Serge Noiraud" <[hidden email]>
To: "Tony Proctor" <[hidden email]>
Cc: "Nick Hall" <[hidden email]>; "Gramps-developers mailing list"
<[hidden email]>
Sent: Sunday, August 11, 2013 4:02 PM
Subject: Re: [Gramps-devel] Filters and regular expressions


> Le 11/08/2013 16:22, Tony Proctor a écrit :
>> I'm not a fan of text-based filters for this type of application. I know
>> through experience that they can be fragile and error-prone, especially
>> if misapplied to an idea that would be safer involving "objects" rather
>> than text.
>>
>> Just to stir things up a little, can I refer you to a FHISO paper that
>> does seem to be relevant here: http://fhiso.org/files/cfp/cfps19.pdf
>
> Perhaps you are right,
>
> But how do you search for search a string  ?
>
> n(e|é|es|ey|oua|oai|o[iya](e|)|a[iy])r(r|)(o(n|t|d|s)|((e|)au(x|t|d|lt|)))
>
> I use this regexp for all the possibilities I found for the name "noiraud"
> since 1515.
>
>>
>>    Tony Proctor
>>
>> ----- Original Message ----- From: "Nick Hall" <[hidden email]>
>> To: "Serge Noiraud" <[hidden email]>
>> Cc: "Gramps-developers mailing list" <[hidden email]>
>> Sent: Sunday, August 11, 2013 3:16 PM
>> Subject: Re: [Gramps-devel] Filters and regular expressions
>>
>>
>>> On 11/08/13 09:59, Serge Noiraud wrote:
>>>> It doesn't work any more for me :
>>>> I get the following messages for all my filters:
>>>>
>>>> WARNING: Too many arguments in filter « XXXXX » !
>>>> Trying to load with subset of arguments.
>>>>
>>>> I needed to edit all the filters and put again "use regexp".
>>>> Now all filters are working again
>>>
>>> Serge,
>>>
>>> Thanks for reporting this.
>>>
>>> The warning occurred with the five upgraded rules that already used
>>> regular expressions.
>>>
>>> People with records containing <substring>
>>> People with the <name>
>>> Families with child with the <Name>
>>> Families with father with the <Name>
>>> Families with mother with the <Name>
>>>
>>> I've added some code to automatically update these rules when they are
>>> first loaded.  Let me know if this fixes your problem.
>>>
>>> Nick.
>>>
>>
>>
>


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Serge Noiraud-2
Le 11/08/2013 17:11, Tony Proctor a écrit :

> With an object-based script, searching is much easier, and much more
> readable and maintainable. It all depends what methods you define on your
> objects.
>
> I haven't looked at that FHISO paper for a while but I think it emphasises
> that a method searching for, say, "John Smith" would automatically apply the
> normal name-matching semantics for the associated software product. This
> means it would treat the name case-blind, possibly accent-blind, and check
> all the alternative names for each person, and handle abbreviations and
> initials too. Hence, in some circumstances, a RegEx wouldn't be necessary
> because the semantics of what you're trying to achieve are provided by a
> supported method.
>
> I'm sure a lot of effort went into your RegEx expression but I'd hate to be
> the person who ended up maintaining it  ;-)
Perhaps it's difficult to maintain, but no automatism could be able to find this regexp !
I think I can simplify it a little, but it will be less readable.
I know it's difficult to read all possiblities from this regexp, but it works correctly for me
but this is the first goal.

Serge

>
>      Tony Proctor
>
>
> ----- Original Message -----
> From: "Serge Noiraud" <[hidden email]>
> To: "Tony Proctor" <[hidden email]>
> Cc: "Nick Hall" <[hidden email]>; "Gramps-developers mailing list"
> <[hidden email]>
> Sent: Sunday, August 11, 2013 4:02 PM
> Subject: Re: [Gramps-devel] Filters and regular expressions
>
>
>> Le 11/08/2013 16:22, Tony Proctor a écrit :
>>> I'm not a fan of text-based filters for this type of application. I know
>>> through experience that they can be fragile and error-prone, especially
>>> if misapplied to an idea that would be safer involving "objects" rather
>>> than text.
>>>
>>> Just to stir things up a little, can I refer you to a FHISO paper that
>>> does seem to be relevant here: http://fhiso.org/files/cfp/cfps19.pdf
>> Perhaps you are right,
>>
>> But how do you search for search a string  ?
>>
>> n(e|é|es|ey|oua|oai|o[iya](e|)|a[iy])r(r|)(o(n|t|d|s)|((e|)au(x|t|d|lt|)))
>>
>> I use this regexp for all the possibilities I found for the name "noiraud"
>> since 1515.
>>
>>>     Tony Proctor
>>>
>>> ----- Original Message ----- From: "Nick Hall" <[hidden email]>
>>> To: "Serge Noiraud" <[hidden email]>
>>> Cc: "Gramps-developers mailing list" <[hidden email]>
>>> Sent: Sunday, August 11, 2013 3:16 PM
>>> Subject: Re: [Gramps-devel] Filters and regular expressions
>>>
>>>
>>>> On 11/08/13 09:59, Serge Noiraud wrote:
>>>>> It doesn't work any more for me :
>>>>> I get the following messages for all my filters:
>>>>>
>>>>> WARNING: Too many arguments in filter « XXXXX » !
>>>>> Trying to load with subset of arguments.
>>>>>
>>>>> I needed to edit all the filters and put again "use regexp".
>>>>> Now all filters are working again
>>>> Serge,
>>>>
>>>> Thanks for reporting this.
>>>>
>>>> The warning occurred with the five upgraded rules that already used
>>>> regular expressions.
>>>>
>>>> People with records containing <substring>
>>>> People with the <name>
>>>> Families with child with the <Name>
>>>> Families with father with the <Name>
>>>> Families with mother with the <Name>
>>>>
>>>> I've added some code to automatically update these rules when they are
>>>> first loaded.  Let me know if this fixes your problem.
>>>>
>>>> Nick.
>>>>


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

Doug
In reply to this post by ACProctor
On 11/08/13 16:11, Tony Proctor wrote:

> With an object-based script, searching is much easier, and much more
> readable and maintainable. It all depends what methods you define on your
> objects.
>
> I haven't looked at that FHISO paper for a while but I think it emphasises
> that a method searching for, say, "John Smith" would automatically apply the
> normal name-matching semantics for the associated software product. This
> means it would treat the name case-blind, possibly accent-blind, and check
> all the alternative names for each person, and handle abbreviations and
> initials too. Hence, in some circumstances, a RegEx wouldn't be necessary
> because the semantics of what you're trying to achieve are provided by a
> supported method.
>
> I'm sure a lot of effort went into your RegEx expression but I'd hate to be
> the person who ended up maintaining it  ;-)
>
>      Tony Proctor
>
>
<snip>

I don't understand the point about maintaining: I think of
filters as essentially one-off, disposable, a powerful way
of solving particular, probably temporary problems and only
rarely having longer-term value; so the question of
maintaining them never really arises, at least for me.

Doug


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

ACProctor

----- Original Message -----
From: "doug" <[hidden email]>
To: "Tony Proctor" <[hidden email]>; "Gramps-developers mailing list"
<[hidden email]>
Sent: Monday, August 12, 2013 12:19 PM
Subject: Re: [Gramps-devel] Filters and regular expressions


> On 11/08/13 16:11, Tony Proctor wrote:
>> With an object-based script, searching is much easier, and much more
>> readable and maintainable. It all depends what methods you define on your
>> objects.
>>
>> I haven't looked at that FHISO paper for a while but I think it
>> emphasises
>> that a method searching for, say, "John Smith" would automatically apply
>> the
>> normal name-matching semantics for the associated software product. This
>> means it would treat the name case-blind, possibly accent-blind, and
>> check
>> all the alternative names for each person, and handle abbreviations and
>> initials too. Hence, in some circumstances, a RegEx wouldn't be necessary
>> because the semantics of what you're trying to achieve are provided by a
>> supported method.
>>
>> I'm sure a lot of effort went into your RegEx expression but I'd hate to
>> be
>> the person who ended up maintaining it  ;-)
>>
>>      Tony Proctor
>>
>>
> <snip>
>
> I don't understand the point about maintaining: I think of filters as
> essentially one-off, disposable, a powerful way of solving particular,
> probably temporary problems and only rarely having longer-term value; so
> the question of maintaining them never really arises, at least for me.
>
> Doug
>

If, as in the original example of this thread, you wanted to filter on all
the known variations of a particular surname then sooner-or-later you're
going to find another variation. Hence, there will be some maintenance
required to adjust your filter.

A stronger point, though (IMHO), is that this information is either already
part of your data, or intrinsic to the functionality of your product. I
cannot understand why you would want a totally independent method of
performing the same thing - one that could, as a result, deliver different
data.

For instance, should we worry about the filter being case-blind? This is
fairly easy to accomplish in RegEx but then what about accent-blind matches?
What about composed and uncomposed Unicode forms? This would all be built
into a good product and an object-based approached could leverage it.
Similarly with the handling of abbreviations (e.g. Chas. for Charles),
diminutive forms (e.g. Tony for Anthony), or initials. Similarly again with
a person who is recorded with multiple names, or names in different
languages.

In effect, the code within the product could make a much better job of
performing your searches/filters/matches/lookups then any RegEx, so why not
provide method calls to take advantage of it?

    Tony Proctor



------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Filters and regular expressions

John Ralls-2

On Aug 12, 2013, at 4:34 AM, Tony Proctor <[hidden email]> wrote:

>
> ----- Original Message -----
> From: "doug" <[hidden email]>
> To: "Tony Proctor" <[hidden email]>; "Gramps-developers mailing list"
> <[hidden email]>
> Sent: Monday, August 12, 2013 12:19 PM
> Subject: Re: [Gramps-devel] Filters and regular expressions
>
>
>> On 11/08/13 16:11, Tony Proctor wrote:
>>> With an object-based script, searching is much easier, and much more
>>> readable and maintainable. It all depends what methods you define on your
>>> objects.
>>>
>>> I haven't looked at that FHISO paper for a while but I think it
>>> emphasises
>>> that a method searching for, say, "John Smith" would automatically apply
>>> the
>>> normal name-matching semantics for the associated software product. This
>>> means it would treat the name case-blind, possibly accent-blind, and
>>> check
>>> all the alternative names for each person, and handle abbreviations and
>>> initials too. Hence, in some circumstances, a RegEx wouldn't be necessary
>>> because the semantics of what you're trying to achieve are provided by a
>>> supported method.
>>>
>>> I'm sure a lot of effort went into your RegEx expression but I'd hate to
>>> be
>>> the person who ended up maintaining it  ;-)
>>>
>>>     Tony Proctor
>>>
>>>
>> <snip>
>>
>> I don't understand the point about maintaining: I think of filters as
>> essentially one-off, disposable, a powerful way of solving particular,
>> probably temporary problems and only rarely having longer-term value; so
>> the question of maintaining them never really arises, at least for me.
>>
>> Doug
>>
>
> If, as in the original example of this thread, you wanted to filter on all
> the known variations of a particular surname then sooner-or-later you're
> going to find another variation. Hence, there will be some maintenance
> required to adjust your filter.
>
> A stronger point, though (IMHO), is that this information is either already
> part of your data, or intrinsic to the functionality of your product. I
> cannot understand why you would want a totally independent method of
> performing the same thing - one that could, as a result, deliver different
> data.
>
> For instance, should we worry about the filter being case-blind? This is
> fairly easy to accomplish in RegEx but then what about accent-blind matches?
> What about composed and uncomposed Unicode forms? This would all be built
> into a good product and an object-based approached could leverage it.
> Similarly with the handling of abbreviations (e.g. Chas. for Charles),
> diminutive forms (e.g. Tony for Anthony), or initials. Similarly again with
> a person who is recorded with multiple names, or names in different
> languages.
>
> In effect, the code within the product could make a much better job of
> performing your searches/filters/matches/lookups then any RegEx, so why not
> provide method calls to take advantage of it?
>

Gramps is an open-source project. Patches welcome.

Regards,
John Ralls


------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
123