icu5.8 pyicu 1.9.5 integration problem (or maybe just stupid user error)
Created by: maveryKearney
I am unable to get python 3.6.0/icu 58.2/pyicu 1.9.5 to work as I expect. Specifically the problem is with word break iteration.
I am running on Linux/mint/rosa (64-bit) and am current with updates. I customize word.txt and word_POSIX.txt files for my ICU work.
The "failure" is that I don't see the results of the word_POSIX.txt customizations. It appears as if the default .so are being used rather than that specified in LD_LIBRARY_PATH.
I've attached two files that illustrate the problem: - word_POSIX.txt with two lines added that have worked on priori versions of ICU and PyICU (search for XmlMarkup) word_POSIX.txt
- icu1.py a short script that demonstrates the problem derived from PyICU tests
[icu1.py.txt](https://github.com/ovalhub/pyicu/files/711907/icu1.py.txt)
At the end of this note I have attached output from icu1.py using ICU 57.1 and pyicu 1.9.3 which show correct execution. And, output from the same script ICU 58.2 and pyicu 1.9.5 which show incorrect execution.
Python building
I am confident that python is not the problem. My customizations work with python 3.6.0, icu 57.1, and pyicu 1.9.3, and several versions prior to that.
ICU building
ICU 58.2 builds without error, and passes its own tests. I build with the commands: make clean ./runConfigureICU make and to test: make check
For reasons I explain below, with 58.2 I also execute "make install" but it doesn't help.
In addition, when I add my customizations, icu 58.2 still builds without error and the icu tests show differences that I expect. For this reason, my "failure" feels more like user error than a genuine bug. My customized data is lurking somewhere. It feels like I am just missing a switch, or environment variable to access it.
PyICU building
When I start working with pyicu 1.9.5, odd things start to happen. The pyicu build commands I use are python setup.py build python setup.py install
The logs below show how I run the icu1.py script which show the results.
Details follow
First, "setup.py build" execution fails. The script reports that it is unable to find the icu *.so files in /usr/local/lib. Specifically the message from setup.py is:
### icu-config: Can't find /usr/local/lib/libicuuc.so - ICU prefix is wrong.
### Try the --prefix= option
### or --detect-prefix
### (If you want to disable this check, use the --noverify option)
### icu-config: Exitting.
Traceback (most recent call last):
File "setup.py", line 11, in <module>
ICU_VERSION = subprocess.check_output(('icu-config', '--version')).strip()
File "/home/mkearney/bryllyg/opt/python_/3_6_0/Lib/subprocess.py", line 336, in check_output
**kwargs).stdout
File "/home/mkearney/bryllyg/opt/python_/3_6_0/Lib/subprocess.py", line 418, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '('icu-config', '--version')' returned non-zero exit status 2.
Indeed they are not there, I have not installed icu yet. I keep normally new versions out of that area until they are proven, and defined LD_LIBRARY_PATH to be .../icu_58_2/source/lib which gets created when ICU is built. This strategy has worked until ICU 58.2
Hoping to isolate the problem, I performed icu "make install" and saw that the icu libraries were installed in /usr/local/lib. That enable me to run pyicu "setup.py build" and "install" However, when I run my liveness test for my customizations, the results I get are those of no customizations installed. Perhaps I am getting the default ICU data.
Bottom line, I can proceed for now with python 3.6.0/icu 57.1/pyicu 1.9.3. FYI, I tried icu 58.2 with pyicu 1.9.3, and got undefined symbol errors, and didn't proceed beyond that point.
Expected output --------- with ICU 57.1 and PyIcu 1.9.3
... /opt/python_/3_6_0_x/bin/python icu1.py DYLD_LIBRARY_PATH not defined. LD_LIBRARY_PATH: .../opt/icu_/57_1_brl/source/lib ICU Break Iterator Sample Program C++ Break Iteration in Python Examining: bbb ccc. Ddd eee fff. Word Boundaries... ----- forward: ----------- 0 3 || bbb ccc. Ddd eee fff. 3 4 | |bbb ccc. Ddd eee fff. 4 7 |bbb| ccc. Ddd eee fff. 7 8 bbb| |ccc. Ddd eee fff. 8 11 bbb |ccc|. Ddd eee fff. 11 12 bbb ccc|.| Ddd eee fff. 12 13 bbb ccc.| |Ddd eee fff. 13 17 bbb ccc. ||Ddd eee fff. 17 20 bbb ccc. |Ddd| eee fff. 20 21 bbb ccc. Ddd| |eee fff. 21 24 bbb ccc. Ddd |eee| fff. 24 25 bbb ccc. Ddd eee| |fff. 25 28 bbb ccc. Ddd eee |fff|. 28 29 bbb ccc. Ddd eee fff|.| End C++ Break Iteration in Python .
Failing output - iterator breaks incorrectly. With ICU 58.2 and PyIcu 1.9.5
... /opt/python_/3_6_0_y/bin/python icu1.py DYLD_LIBRARY_PATH not defined. LD_LIBRARY_PATH: .../opt/icu_/58_2_brl/source/lib ICU Break Iterator Sample Program C++ Break Iteration in Python Examining: bbb ccc. Ddd eee fff. Word Boundaries... ----- forward: ----------- 0 1 |<|a> bbb ccc. Ddd eee fff. 1 2 <|a|> bbb ccc. Ddd eee fff. 2 3 <a|>| bbb ccc. Ddd eee fff. 3 4 | |bbb ccc. Ddd eee fff. 4 7 |bbb| ccc. Ddd eee fff. 7 8 bbb| |ccc. Ddd eee fff. 8 11 bbb |ccc|. Ddd eee fff. 11 12 bbb ccc|.| Ddd eee fff. 12 13 bbb ccc.| |Ddd eee fff. 13 14 bbb ccc. |<|/a>Ddd eee fff. 14 15 bbb ccc. <|/|a>Ddd eee fff. 15 16 bbb ccc. </|a|>Ddd eee fff. 16 17 bbb ccc. </a|>|Ddd eee fff. 17 20 bbb ccc. |Ddd| eee fff. 20 21 bbb ccc. Ddd| |eee fff. 21 24 bbb ccc. Ddd |eee| fff. 24 25 bbb ccc. Ddd eee| |fff. 25 28 bbb ccc. Ddd eee |fff|. 28 29 bbb ccc. Ddd eee fff|.| End C++ Break Iteration in Python