Why is Travis CI unable to import coverage and pyicu?

classic Classic list List threaded Threaded
14 messages Options
Reply | Threaded
Open this post in threaded view
|

Why is Travis CI unable to import coverage and pyicu?

Tim Lyons
Administrator
I see in the Travis CI report:


nose.plugins.cover: ERROR: Coverage not available: unable to import coverage module

.gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module named PyICU. Localization will be impaired. Use your package manager to install PyICU


Any idea why neither of these modules are imported? Anybody able to fix this?

(coverage worked about a month ago, and I see that there is special code about retrying pyicu, but apparently that isn't enough).

Regards,
Tim.
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank
On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons <[hidden email]> wrote:
I see in the Travis CI report:


nose.plugins.cover: ERROR: Coverage not available: unable to import coverage
module

.gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
named PyICU. Localization will be impaired. Use your package manager to
install PyICU


Any idea why neither of these modules are imported? Anybody able to fix
this?

(coverage worked about a month ago, and I see that there is special code
about retrying pyicu, but apparently that isn't enough).

Travis isn't configured very well for our needs:

* Gtk, etc.
* Python3

Currently, all of the items installed with pip in .travis.yml are not used, because nosetests3 uses the built-in Python3 which has access to the system installed packages (gtk, cairo, etc). There are two competing Python3's in Travis:

* the Python3 environment versions (3.4 and 3.5), which doesn't have access to system-installed packages (gtk)
* the Python3 system version (3.3 I think), which only has access to older packages (no pyicu)

There is a use-system-modules flag, but it doesn't work with Python3.

I think the root cause is that Travis is based on an older Linux distro. If they make available a newer distro, we could use that. Or, if gtk and friends were available via pip, we could switch to the Python3 environment.

BTW, coverage never has worked via Travis; I have had to manually run nosetests locally, and force it to update coverage. This is because the coverage package available to system Python3 is too old.

-Doug
 

Regards,
Tim.



--
View this message in context: http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124.html
Sent from the GRAMPS - Dev mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

Tim Lyons
Administrator
DS Blank wrote
On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons <[hidden email]> wrote:

> I see in the Travis CI report:
>
>
> nose.plugins.cover: ERROR: Coverage not available: unable to import
> coverage
> module
>
> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
> named PyICU. Localization will be impaired. Use your package manager to
> install PyICU
>
>
> Any idea why neither of these modules are imported? Anybody able to fix
> this?
>
> (coverage worked about a month ago, and I see that there is special code
> about retrying pyicu, but apparently that isn't enough).
>

Travis isn't configured very well for our needs:

* Gtk, etc.
* Python3

Currently, all of the items installed with pip in .travis.yml are not used,
because nosetests3 uses the built-in Python3 which has access to the system
installed packages (gtk, cairo, etc). There are two competing Python3's in
Travis:

* the Python3 environment versions (3.4 and 3.5), which doesn't have access
to system-installed packages (gtk)
* the Python3 system version (3.3 I think), which only has access to older
packages (no pyicu)

There is a use-system-modules flag, but it doesn't work with Python3.

I think the root cause is that Travis is based on an older Linux distro. If
they make available a newer distro, we could use that. Or, if gtk and
friends were available via pip, we could switch to the Python3 environment.

BTW, coverage never has worked via Travis; I have had to manually run
nosetests locally, and force it to update coverage. This is because the
coverage package available to system Python3 is too old.

-Doug
Thanks for the quick reply.

Please note that the Gramps requirement in the README is for Python 3.2, so our Travis tests should be using that, NOT Python 3.4 as it seems to be at the moment.

Why can't we install the needed packages into the Python3 system version?




Just for information, Travis is using Ubuntu 12.04.5 which was released on 7 August 2014. It has an end of life of April 2017.


The next LTS is Ubuntu 14.04.3 LTS Trusty Tahr released 6 August 2015.

Apparently Travis have a beta Trusty environment: "Our Ubuntu 14.04 Trusty Tahr images are based on the ubuntu-1404-trusty-v20150909a Google Compute Engine image, which itself is based on 2015-09-08 revision of 14.04.3".

This is a beta because "Note: the mega image does not currently contain all the common tools and services present in our Precise images. This is something we’ll be expanding over the next couple of months".


Would it be possible to switch to the Trusty beta Build Environment[1] if this would support pyicu and coverage (as well as gtk, cairo and friends)? [1]

Regards,
Tim.



[1] https://docs.travis-ci.com/user/trusty-ci-environment/
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank
On Mon, Jul 11, 2016 at 7:16 AM, Tim Lyons <[hidden email]> wrote:
DS Blank wrote
> On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons &lt;

> guy.linton@

> &gt; wrote:
>
>> I see in the Travis CI report:
>>
>>
>> nose.plugins.cover: ERROR: Coverage not available: unable to import
>> coverage
>> module
>>
>> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
>> named PyICU. Localization will be impaired. Use your package manager to
>> install PyICU
>>
>>
>> Any idea why neither of these modules are imported? Anybody able to fix
>> this?
>>
>> (coverage worked about a month ago, and I see that there is special code
>> about retrying pyicu, but apparently that isn't enough).
>>
>
> Travis isn't configured very well for our needs:
>
> * Gtk, etc.
> * Python3
>
> Currently, all of the items installed with pip in .travis.yml are not
> used,
> because nosetests3 uses the built-in Python3 which has access to the
> system
> installed packages (gtk, cairo, etc). There are two competing Python3's in
> Travis:
>
> * the Python3 environment versions (3.4 and 3.5), which doesn't have
> access
> to system-installed packages (gtk)
> * the Python3 system version (3.3 I think), which only has access to older
> packages (no pyicu)
>
> There is a use-system-modules flag, but it doesn't work with Python3.
>
> I think the root cause is that Travis is based on an older Linux distro.
> If
> they make available a newer distro, we could use that. Or, if gtk and
> friends were available via pip, we could switch to the Python3
> environment.
>
> BTW, coverage never has worked via Travis; I have had to manually run
> nosetests locally, and force it to update coverage. This is because the
> coverage package available to system Python3 is too old.
>
> -Doug

Thanks for the quick reply.

Please note that the Gramps requirement in the README is for Python 3.2, so
our Travis tests should be using that, NOT Python 3.4 as it seems to be at
the moment.

Ideally, we can test Gramps on Travis with all of the versions of Python that we support. So, the tests could theoretically run on 3.2, 3.3, 3.4, 3.5, and beyond.
 

Why can't we install the needed packages into the Python3 system version?

I forget all of the problems, but I couldn't get newer packages (pyicu, coverage, pip) to install with it. 
 

Just for information, Travis is using Ubuntu 12.04.5 which was released on 7
August 2014. It has an end of life of April 2017.

The next LTS is Ubuntu 14.04.3 LTS Trusty Tahr released 6 August 2015.

Apparently Travis have a beta Trusty environment: "Our Ubuntu 14.04 Trusty
Tahr images are based on the ubuntu-1404-trusty-v20150909a Google Compute
Engine image, which itself is based on 2015-09-08 revision of 14.04.3".

This is a beta because "Note: the mega image does not currently contain all
the common tools and services present in our Precise images. This is
something we’ll be expanding over the next couple of months".


Would it be possible to switch to the Trusty beta Build Environment[1] if
this would support pyicu and coverage (as well as gtk, cairo and friends)?
[1]

Perhaps. If someone makes a PR to the .travis.yml, it will run on Travis and we could see if such a change works. 

-Doug
 

Regards,
Tim.



[1] https://docs.travis-ci.com/user/trusty-ci-environment/



--
View this message in context: http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124p4676126.html
Sent from the GRAMPS - Dev mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

prculley
I personally wonder why we are so hot on staying on an old version of Python.  Seems to me we should just say that for v5.x we need at least 3.4.  Especially if we start having Travis run with multiple versions.

Would cut down on the potential for support issues.  And we could update some code to quit getting so many deprecation warnings ;-).

Paul Culley

On Mon, Jul 11, 2016 at 7:08 AM, Doug Blank <[hidden email]> wrote:
On Mon, Jul 11, 2016 at 7:16 AM, Tim Lyons <[hidden email]> wrote:
DS Blank wrote
> On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons &lt;

> guy.linton@

> &gt; wrote:
>
>> I see in the Travis CI report:
>>
>>
>> nose.plugins.cover: ERROR: Coverage not available: unable to import
>> coverage
>> module
>>
>> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
>> named PyICU. Localization will be impaired. Use your package manager to
>> install PyICU
>>
>>
>> Any idea why neither of these modules are imported? Anybody able to fix
>> this?
>>
>> (coverage worked about a month ago, and I see that there is special code
>> about retrying pyicu, but apparently that isn't enough).
>>
>
> Travis isn't configured very well for our needs:
>
> * Gtk, etc.
> * Python3
>
> Currently, all of the items installed with pip in .travis.yml are not
> used,
> because nosetests3 uses the built-in Python3 which has access to the
> system
> installed packages (gtk, cairo, etc). There are two competing Python3's in
> Travis:
>
> * the Python3 environment versions (3.4 and 3.5), which doesn't have
> access
> to system-installed packages (gtk)
> * the Python3 system version (3.3 I think), which only has access to older
> packages (no pyicu)
>
> There is a use-system-modules flag, but it doesn't work with Python3.
>
> I think the root cause is that Travis is based on an older Linux distro.
> If
> they make available a newer distro, we could use that. Or, if gtk and
> friends were available via pip, we could switch to the Python3
> environment.
>
> BTW, coverage never has worked via Travis; I have had to manually run
> nosetests locally, and force it to update coverage. This is because the
> coverage package available to system Python3 is too old.
>
> -Doug

Thanks for the quick reply.

Please note that the Gramps requirement in the README is for Python 3.2, so
our Travis tests should be using that, NOT Python 3.4 as it seems to be at
the moment.

Ideally, we can test Gramps on Travis with all of the versions of Python that we support. So, the tests could theoretically run on 3.2, 3.3, 3.4, 3.5, and beyond.
 

Why can't we install the needed packages into the Python3 system version?

I forget all of the problems, but I couldn't get newer packages (pyicu, coverage, pip) to install with it. 
 

Just for information, Travis is using Ubuntu 12.04.5 which was released on 7
August 2014. It has an end of life of April 2017.

The next LTS is Ubuntu 14.04.3 LTS Trusty Tahr released 6 August 2015.

Apparently Travis have a beta Trusty environment: "Our Ubuntu 14.04 Trusty
Tahr images are based on the ubuntu-1404-trusty-v20150909a Google Compute
Engine image, which itself is based on 2015-09-08 revision of 14.04.3".

This is a beta because "Note: the mega image does not currently contain all
the common tools and services present in our Precise images. This is
something we’ll be expanding over the next couple of months".


Would it be possible to switch to the Trusty beta Build Environment[1] if
this would support pyicu and coverage (as well as gtk, cairo and friends)?
[1]

Perhaps. If someone makes a PR to the .travis.yml, it will run on Travis and we could see if such a change works. 

-Doug
 

Regards,
Tim.



[1] https://docs.travis-ci.com/user/trusty-ci-environment/



--
View this message in context: http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124p4676126.html
Sent from the GRAMPS - Dev mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel



------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank
On Mon, Jul 11, 2016 at 10:36 AM, Paul Culley <[hidden email]> wrote:
I personally wonder why we are so hot on staying on an old version of Python.  Seems to me we should just say that for v5.x we need at least 3.4.  Especially if we start having Travis run with multiple versions.

Would cut down on the potential for support issues.  And we could update some code to quit getting so many deprecation warnings ;-).

Generally, we only require a later version of Python if there is something that we want to rely on. Otherwise, we are upping the requirements, and perhaps putting pressure on people to upgrade their Python versions (perhaps on older, non-updatable machines).

But, I agree, that Python 3.2 is somewhat arbitrary, and I'd be glad to see us target later versions on a scheduled plan. 

-Doug

 


Paul Culley

On Mon, Jul 11, 2016 at 7:08 AM, Doug Blank <[hidden email]> wrote:
On Mon, Jul 11, 2016 at 7:16 AM, Tim Lyons <[hidden email]> wrote:
DS Blank wrote
> On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons &lt;

> guy.linton@

> &gt; wrote:
>
>> I see in the Travis CI report:
>>
>>
>> nose.plugins.cover: ERROR: Coverage not available: unable to import
>> coverage
>> module
>>
>> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
>> named PyICU. Localization will be impaired. Use your package manager to
>> install PyICU
>>
>>
>> Any idea why neither of these modules are imported? Anybody able to fix
>> this?
>>
>> (coverage worked about a month ago, and I see that there is special code
>> about retrying pyicu, but apparently that isn't enough).
>>
>
> Travis isn't configured very well for our needs:
>
> * Gtk, etc.
> * Python3
>
> Currently, all of the items installed with pip in .travis.yml are not
> used,
> because nosetests3 uses the built-in Python3 which has access to the
> system
> installed packages (gtk, cairo, etc). There are two competing Python3's in
> Travis:
>
> * the Python3 environment versions (3.4 and 3.5), which doesn't have
> access
> to system-installed packages (gtk)
> * the Python3 system version (3.3 I think), which only has access to older
> packages (no pyicu)
>
> There is a use-system-modules flag, but it doesn't work with Python3.
>
> I think the root cause is that Travis is based on an older Linux distro.
> If
> they make available a newer distro, we could use that. Or, if gtk and
> friends were available via pip, we could switch to the Python3
> environment.
>
> BTW, coverage never has worked via Travis; I have had to manually run
> nosetests locally, and force it to update coverage. This is because the
> coverage package available to system Python3 is too old.
>
> -Doug

Thanks for the quick reply.

Please note that the Gramps requirement in the README is for Python 3.2, so
our Travis tests should be using that, NOT Python 3.4 as it seems to be at
the moment.

Ideally, we can test Gramps on Travis with all of the versions of Python that we support. So, the tests could theoretically run on 3.2, 3.3, 3.4, 3.5, and beyond.
 

Why can't we install the needed packages into the Python3 system version?

I forget all of the problems, but I couldn't get newer packages (pyicu, coverage, pip) to install with it. 
 

Just for information, Travis is using Ubuntu 12.04.5 which was released on 7
August 2014. It has an end of life of April 2017.

The next LTS is Ubuntu 14.04.3 LTS Trusty Tahr released 6 August 2015.

Apparently Travis have a beta Trusty environment: "Our Ubuntu 14.04 Trusty
Tahr images are based on the ubuntu-1404-trusty-v20150909a Google Compute
Engine image, which itself is based on 2015-09-08 revision of 14.04.3".

This is a beta because "Note: the mega image does not currently contain all
the common tools and services present in our Precise images. This is
something we’ll be expanding over the next couple of months".


Would it be possible to switch to the Trusty beta Build Environment[1] if
this would support pyicu and coverage (as well as gtk, cairo and friends)?
[1]

Perhaps. If someone makes a PR to the .travis.yml, it will run on Travis and we could see if such a change works. 

-Doug
 

Regards,
Tim.



[1] https://docs.travis-ci.com/user/trusty-ci-environment/



--
View this message in context: http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124p4676126.html
Sent from the GRAMPS - Dev mailing list archive at Nabble.com.

------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel



------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel



------------------------------------------------------------------------------
Attend Shape: An AT&T Tech Expo July 15-16. Meet us at AT&T Park in San
Francisco, CA to explore cutting-edge tech and listen to tech luminaries
present their vision of the future. This family event has something for
everyone, including kids. Get more information and register today.
http://sdm.link/attshape
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

Tim Lyons
Administrator
In reply to this post by DS Blank
DS Blank wrote
On Mon, Jul 11, 2016 at 7:16 AM, Tim Lyons <[hidden email]> wrote:

> DS Blank wrote
> > On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons <
>
> > guy.linton@
>
> > > wrote:
> >
> >> I see in the Travis CI report:
> >>
> >>
> >> nose.plugins.cover: ERROR: Coverage not available: unable to import
> >> coverage
> >> module
> >>
> >> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
> >> named PyICU. Localization will be impaired. Use your package manager to
> >> install PyICU
> >>
> >>
> >> Any idea why neither of these modules are imported? Anybody able to fix
> >> this?
> >>
> >> (coverage worked about a month ago, and I see that there is special code
> >> about retrying pyicu, but apparently that isn't enough).
> >>
> >
> > Travis isn't configured very well for our needs:
> >
> > * Gtk, etc.
> > * Python3
> >
> > Currently, all of the items installed with pip in .travis.yml are not
> > used,
> > because nosetests3 uses the built-in Python3 which has access to the
> > system
> > installed packages (gtk, cairo, etc). There are two competing Python3's
> in
> > Travis:
> >
> > * the Python3 environment versions (3.4 and 3.5), which doesn't have
> > access
> > to system-installed packages (gtk)
> > * the Python3 system version (3.3 I think), which only has access to
> older
> > packages (no pyicu)
> >
> > There is a use-system-modules flag, but it doesn't work with Python3.
> >
> > I think the root cause is that Travis is based on an older Linux distro.
> > If
> > they make available a newer distro, we could use that. Or, if gtk and
> > friends were available via pip, we could switch to the Python3
> > environment.
> >
> > BTW, coverage never has worked via Travis; I have had to manually run
> > nosetests locally, and force it to update coverage. This is because the
> > coverage package available to system Python3 is too old.
> >
> > -Doug
>
> Thanks for the quick reply.
>
> Please note that the Gramps requirement in the README is for Python 3.2, so
> our Travis tests should be using that, NOT Python 3.4 as it seems to be at
> the moment.
>

Ideally, we can test Gramps on Travis with all of the versions of Python
that we support. So, the tests could theoretically run on 3.2, 3.3, 3.4,
3.5, and beyond.


>
> Why can't we install the needed packages into the Python3 system version?
>

I forget all of the problems, but I couldn't get newer packages (pyicu,
coverage, pip) to install with it.

I have created a travis branch on github to experiment with fixing the problems.

(1) I have added python-pyicu, but this is not on the Travis whitelist of allowed packages. I have made a request to travis to get it added.

(2) I tried changing to Python 3.2, but coverage.py would not compile (see below), so changed to 3.3.

(3) I have added pip install of codecov, since this just seems to be a script, and I think is not affected by which python is in use.

(3) I added building coverage from source, and the build and install seems to work (it fails with Python 3.2 because of u'foo'). I have added the path to where coverage.py is installed to PYTHONPATH for nosetests3. However, when nosetests3 runs, it seems as though it is calling Python 3.2 instead of the python I requested.

Help please - I have run out of ideas.

The latest build is here:

https://travis-ci.org/gramps-project/gramps/builds/149049184

Tim.
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank

On Mon, Aug 1, 2016 at 7:41 PM, Tim Lyons [via GRAMPS] <[hidden email]> wrote:
DS Blank wrote
On Mon, Jul 11, 2016 at 7:16 AM, Tim Lyons <[hidden email]> wrote:

> DS Blank wrote
> > On Mon, Jul 11, 2016 at 6:02 AM, Tim Lyons <
>
> > guy.linton@
>

> > > wrote:
> >
> >> I see in the Travis CI report:
> >>
> >>
> >> nose.plugins.cover: ERROR: Coverage not available: unable to import
> >> coverage
> >> module
> >>
> >> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
> >> named PyICU. Localization will be impaired. Use your package manager to
> >> install PyICU
> >>
> >>
> >> Any idea why neither of these modules are imported? Anybody able to fix
> >> this?
> >>
> >> (coverage worked about a month ago, and I see that there is special code
> >> about retrying pyicu, but apparently that isn't enough).
> >>
> >
> > Travis isn't configured very well for our needs:
> >
> > * Gtk, etc.
> > * Python3
> >
> > Currently, all of the items installed with pip in .travis.yml are not
> > used,
> > because nosetests3 uses the built-in Python3 which has access to the
> > system
> > installed packages (gtk, cairo, etc). There are two competing Python3's
> in
> > Travis:
> >
> > * the Python3 environment versions (3.4 and 3.5), which doesn't have
> > access
> > to system-installed packages (gtk)
> > * the Python3 system version (3.3 I think), which only has access to
> older
> > packages (no pyicu)
> >
> > There is a use-system-modules flag, but it doesn't work with Python3.
> >
> > I think the root cause is that Travis is based on an older Linux distro.
> > If
> > they make available a newer distro, we could use that. Or, if gtk and
> > friends were available via pip, we could switch to the Python3
> > environment.
> >
> > BTW, coverage never has worked via Travis; I have had to manually run
> > nosetests locally, and force it to update coverage. This is because the
> > coverage package available to system Python3 is too old.
> >
> > -Doug
>
> Thanks for the quick reply.
>
> Please note that the Gramps requirement in the README is for Python 3.2, so
> our Travis tests should be using that, NOT Python 3.4 as it seems to be at
> the moment.
>

Ideally, we can test Gramps on Travis with all of the versions of Python
that we support. So, the tests could theoretically run on 3.2, 3.3, 3.4,
3.5, and beyond.


>
> Why can't we install the needed packages into the Python3 system version?
>

I forget all of the problems, but I couldn't get newer packages (pyicu,
coverage, pip) to install with it.

I have created a travis branch on github to experiment with fixing the problems.

(1) I have added python-pyicu, but this is not on the Travis whitelist of allowed packages. I have made a request to travis to get it added.

(2) I tried changing to Python 3.2, but coverage.py would not compile (see below), so changed to 3.3.

(3) I have added pip install of codecov, since this just seems to be a script, and I think is not affected by which python is in use.

(3) I added building coverage from source, and the build and install seems to work (it fails with Python 3.2 because of u'foo'). I have added the path to where coverage.py is installed to PYTHONPATH for nosetests3. However, when nosetests3 runs, it seems as though it is calling Python 3.2 instead of the python I requested.

Help please - I have run out of ideas.

Excellent work!

I think you are right on (3). But I think you can forget about nosetests3, and run the tests with some variation of "/path/to/python3.3 -m unittest discover ..." I'm not sure exactly what it would be, but I think you are close.

-Doug
 

The latest build is here:

https://travis-ci.org/gramps-project/gramps/builds/149049184

Tim.


If you reply to this email, your message will be added to the discussion below:
http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124p4676389.html
This email was sent by Tim Lyons (via Nabble)
To receive all replies by email, subscribe to this discussion

Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

Tim Lyons
Administrator
DS Blank wrote
> (3) I added building coverage from source, and the build and install seems
> to work (it fails with Python 3.2 because of u'foo'). I have added the path
> to where coverage.py is installed to PYTHONPATH for nosetests3. However,
> when nosetests3 runs, it seems as though it is calling Python 3.2 instead
> of the python I requested.
>
> Help please - I have run out of ideas.
>

Excellent work!
Thanks
DS Blank wrote
I think you are right on (3). But I think you can forget about nosetests3,
and run the tests with some variation of "/path/to/python3.3 -m unittest
discover ..." I'm not sure exactly what it would be, but I think you are
close.

Yes but (a) that wouldn't support coverage, and (b) unitest discover doesn't support exclude.

Tim
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank
On Tue, Aug 2, 2016 at 1:01 PM, Tim Lyons [via GRAMPS] <[hidden email]> wrote:
DS Blank wrote
> (3) I added building coverage from source, and the build and install seems
> to work (it fails with Python 3.2 because of u'foo'). I have added the path
> to where coverage.py is installed to PYTHONPATH for nosetests3. However,
> when nosetests3 runs, it seems as though it is calling Python 3.2 instead
> of the python I requested.
>
> Help please - I have run out of ideas.
>

Excellent work!
Thanks
DS Blank wrote
I think you are right on (3). But I think you can forget about nosetests3,
and run the tests with some variation of "/path/to/python3.3 -m unittest
discover ..." I'm not sure exactly what it would be, but I think you are
close.

Yes but (a) that wouldn't support coverage, and (b) unitest discover doesn't support exclude.

Perhaps http://doc.pytest.org/ can fill in for plain unittest. We can rename the files to be excluded, if necessary.

-Doug
 

Tim


If you reply to this email, your message will be added to the discussion below:
http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124p4676402.html
This email was sent by Tim Lyons (via Nabble)
To receive all replies by email, subscribe to this discussion

Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

Tim Lyons
Administrator
In reply to this post by Tim Lyons
Tim Lyons wrote
I see in the Travis CI report:


nose.plugins.cover: ERROR: Coverage not available: unable to import coverage module

.gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module named PyICU. Localization will be impaired. Use your package manager to install PyICU


Any idea why neither of these modules are imported? Anybody able to fix this?

I have fixed this all in commit

https://github.com/gramps-project/gramps/commit/6245d219a93e9dafa001b8699fc5185f7778c300


There are still some omissions which I have noted in FIXMEs, but the basic testing and coverage now work fine.

I think I now understand a bit more about the difference between the virtualenv (which is no good for us; although it has a variety of Python versions available, pip doesn't get the packages we need) and the underlying Ubuntu environment (which is what we use; this only has an older Python3 in the normal Precise Ubuntu 12.04.5, which isn't modern enough to compile coverage, and doesn't have a python3 version of PyICU; so I have switched to Ubuntu Trusty).

I tried for some time to mix the environments, but this didn't work - I don't understand this - does the virtualenv have some kind of barrier which prevents access in either direction between the virtualenv and the normal one?

I plan to have a look at the FIXMEs.

Tim.
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank
On Wed, Aug 17, 2016 at 6:52 AM, Tim Lyons <[hidden email]> wrote:
Tim Lyons wrote
> I see in the Travis CI report:
>
>
> nose.plugins.cover: ERROR: Coverage not available: unable to import
> coverage module
>
> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
> named PyICU. Localization will be impaired. Use your package manager to
> install PyICU
>
>
> Any idea why neither of these modules are imported? Anybody able to fix
> this?


I have fixed this all in commit

https://github.com/gramps-project/gramps/commit/6245d219a93e9dafa001b8699fc5185f7778c300

Much appreciated! That looks like a solid method to move forward. Now, coverage reports are updated automatically. We are up to 38.12% (after starting at 21.52%). Now we can test the pyicu code as well.


But we have a long way to go on the testing front to reach a reasonable 80%. Much of that is GUI code which will need some type of framework (mock, something gtk-specific, or framebuffer-based, etc).

-Doug

 



There are still some omissions which I have noted in FIXMEs, but the basic
testing and coverage now work fine.

I think I now understand a bit more about the difference between the
virtualenv (which is no good for us; although it has a variety of Python
versions available, pip doesn't get the packages we need) and the underlying
Ubuntu environment (which is what we use; this only has an older Python3 in
the normal Precise Ubuntu 12.04.5, which isn't modern enough to compile
coverage, and doesn't have a python3 version of PyICU; so I have switched to
Ubuntu Trusty).

I tried for some time to mix the environments, but this didn't work - I
don't understand this - does the virtualenv have some kind of barrier which
prevents access in either direction between the virtualenv and the normal
one?

I plan to have a look at the FIXMEs.

Tim.



--
View this message in context: http://gramps.1791082.n4.nabble.com/Why-is-Travis-CI-unable-to-import-coverage-and-pyicu-tp4676124p4676489.html
Sent from the GRAMPS - Dev mailing list archive at Nabble.com.

------------------------------------------------------------------------------
_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel


------------------------------------------------------------------------------

_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

John Ralls-2

On Aug 17, 2016, at 4:09 AM, Doug Blank <[hidden email]> wrote:

On Wed, Aug 17, 2016 at 6:52 AM, Tim Lyons <[hidden email]> wrote:
Tim Lyons wrote
> I see in the Travis CI report:

>
>
> nose.plugins.cover: ERROR: Coverage not available: unable to import
> coverage module
>
> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
> named PyICU. Localization will be impaired. Use your package manager to
> install PyICU
>
>
> Any idea why neither of these modules are imported? Anybody able to fix
> this?


I have fixed this all in commit

https://github.com/gramps-project/gramps/commit/6245d219a93e9dafa001b8699fc5185f7778c300

Much appreciated! That looks like a solid method to move forward. Now, coverage reports are updated automatically. We are up to 38.12% (after starting at 21.52%). Now we can test the pyicu code as well.


But we have a long way to go on the testing front to reach a reasonable 80%. Much of that is GUI code which will need some type of framework (mock, something gtk-specific, or framebuffer-based, etc).

Automated GUI testing is difficult and generally fragile. IMO there's not a lot of point to it outside of the GUI framework itself.

A better approach is to ensure that we strictly comply with MVC separation so that the only thing the view code does is call controller functions with no manipulation of the results. That makes for what some might consider excessive back-and-forth in cases like the date validity checks and anything involving GtkTreeView/GtkTreeModel. The benefit is that if there's no code worth testing in the GUI directories then we can exclude them from the coverage computation and be sure that we really do have good coverage of the important code.

Regards,
John Ralls

------------------------------------------------------------------------------

_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel
Reply | Threaded
Open this post in threaded view
|

Re: Why is Travis CI unable to import coverage and pyicu?

DS Blank
On Wed, Aug 17, 2016 at 10:01 AM, John Ralls <[hidden email]> wrote:

On Aug 17, 2016, at 4:09 AM, Doug Blank <[hidden email]> wrote:

On Wed, Aug 17, 2016 at 6:52 AM, Tim Lyons <[hidden email]> wrote:
Tim Lyons wrote
> I see in the Travis CI report:

>
>
> nose.plugins.cover: ERROR: Coverage not available: unable to import
> coverage module
>
> .gramps.gen.utils.grampslocale.WARNING: ICU not loaded because No module
> named PyICU. Localization will be impaired. Use your package manager to
> install PyICU
>
>
> Any idea why neither of these modules are imported? Anybody able to fix
> this?


I have fixed this all in commit

https://github.com/gramps-project/gramps/commit/6245d219a93e9dafa001b8699fc5185f7778c300

Much appreciated! That looks like a solid method to move forward. Now, coverage reports are updated automatically. We are up to 38.12% (after starting at 21.52%). Now we can test the pyicu code as well.


But we have a long way to go on the testing front to reach a reasonable 80%. Much of that is GUI code which will need some type of framework (mock, something gtk-specific, or framebuffer-based, etc).

Automated GUI testing is difficult and generally fragile. IMO there's not a lot of point to it outside of the GUI framework itself.

A better approach is to ensure that we strictly comply with MVC separation so that the only thing the view code does is call controller functions with no manipulation of the results. That makes for what some might consider excessive back-and-forth in cases like the date validity checks and anything involving GtkTreeView/GtkTreeModel. The benefit is that if there's no code worth testing in the GUI directories then we can exclude them from the coverage computation and be sure that we really do have good coverage of the important code.

I agree with this philosophy in general---I meant developing tests that exercise the logic in the GUI code, not testing the actual GUI itself. But we can test a lot of logic by instantiating and calling methods in existing GUI classes, by simply mocking a few gtk methods (such as Window.show). 

-Doug
 

Regards,
John Ralls


------------------------------------------------------------------------------

_______________________________________________
Gramps-devel mailing list
[hidden email]
https://lists.sourceforge.net/lists/listinfo/gramps-devel