Anzeige:
Ergebnis 1 bis 9 von 9

Thema: Python pyarrow not found when install with pip

  1. #1
    Registrierter Benutzer
    Registriert seit
    Aug 2017
    Beiträge
    15

    Red face Python pyarrow not found when install with pip

    Hello,
    i have installed pyarrow with pip and it was not found by python.
    My python-programm was running in Pycharm with installed pyarrow well.

    I thing that a path was not correct , but i cant find it.

    Statement with error:
    df = pd.read_csv(
    CSV_FILENAME,
    engine="pyarrow",
    header=0,
    sep=";",
    usecols=["DateTime", "Verdichter_laeuft_seit", "Temp_EQ_Eintritt"],
    parse_dates=["DateTime"],
    dtype={"Verdichter_laeuft_seit": "float", "Temp_EQ_Eintritt": "float"}
    )
    Traceback (most recent call last):
    File "/home/user/./plotwp-test.py", line 54, in <module>
    main()
    File "/home/user/./plotwp-test.py", line 28, in main
    df = pd.read_csv(
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 688, in read_csv
    return _read(filepath_or_buffer, kwds)
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 454, in _read
    parser = TextFileReader(fp_or_buf, **kwds)
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 948, in __init__
    self._make_engine(self.engine)
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 1187, in _make_engine
    raise ValueError(
    ValueError: Unknown engine: pyarrow (valid options are "c", "python", or "python-fwf")
    user@mx:~
    $ pip install pyarrow
    Defaulting to user installation because normal site-packages is not writeable
    Collecting pyarrow
    Using cached pyarrow-14.0.2-cp39-cp39-manylinux_2_28_x86_64.whl (38.0 MB)
    Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.9/dist-packages (from pyarrow) (1.26.2)
    Installing collected packages: pyarrow
    Successfully installed pyarrow-14.0.2
    whatis wrong ?

  2. #2
    Registrierter Benutzer
    Registriert seit
    Apr 2009
    Ort
    Erde
    Beiträge
    2.804
    You should start to use venvs, maybe pycharm already did that for you and you have to start the venv if you want to launch your application without pycharm
    Gruß nopes
    (,,,)---(^.^)---(,,,) /var/log/messages | grep cat

  3. #3
    Registrierter Benutzer
    Registriert seit
    Aug 2017
    Beiträge
    15
    I would use pyarrow not only for one project. Is ist possible to install it global ?

  4. #4
    Registrierter Benutzer
    Registriert seit
    Apr 2009
    Ort
    Erde
    Beiträge
    2.804
    Yes this is possible, running pip outside an venv and as root should do that.

    You might want to check if you have installed python2 on you system (fe debian still do this [at least last time I checked]) in this case you might have to use pip3 instead of pip (fe at debian systems)
    Gruß nopes
    (,,,)---(^.^)---(,,,) /var/log/messages | grep cat

  5. #5
    Registrierter Benutzer
    Registriert seit
    Aug 2017
    Beiträge
    15
    It does not help!

    $ sudo pip3 install pyarrow
    Collecting pyarrow
    Downloading pyarrow-14.0.2-cp39-cp39-manylinux_2_28_x86_64.whl (38.0 MB)
    |████████████████████████████████| 38.0 MB 6.3 MB/s
    Requirement already satisfied: numpy>=1.16.6 in /usr/local/lib/python3.9/dist-packages (from pyarrow) (1.26.2)
    Installing collected packages: pyarrow
    Successfully installed pyarrow-14.0.2
    user@mx:~
    $ ./plotwp-test.py
    Traceback (most recent call last):
    File "/home/user/./plotwp-test.py", line 54, in <module>
    main()
    File "/home/user/./plotwp-test.py", line 28, in main
    df = pd.read_csv(
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 688, in read_csv
    return _read(filepath_or_buffer, kwds)
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 454, in _read
    parser = TextFileReader(fp_or_buf, **kwds)
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 948, in __init__
    self._make_engine(self.engine)
    File "/usr/lib/python3/dist-packages/pandas/io/parsers.py", line 1187, in _make_engine
    raise ValueError(
    ValueError: Unknown engine: pyarrow (valid options are "c", "python", or "python-fwf")

  6. #6
    Registrierter Benutzer Avatar von Huhn Hur Tu
    Registriert seit
    Nov 2003
    Ort
    Karlsruhe
    Beiträge
    2.238
    please check if your pyarrow package is in the $PATH by

    find /usr -name "pyarrow*"
    Zwischen anonym sein wollen und seine Daten nicht verkaufen wollen, liegen zwei Welten. Wenn man sich einen kostenpflichtigen Dienst sucht, dann meist, weil man für diese Dienstleistung zahlt und nicht selbst das Produkt sein will.


  7. #7
    Registrierter Benutzer Avatar von Huhn Hur Tu
    Registriert seit
    Nov 2003
    Ort
    Karlsruhe
    Beiträge
    2.238
    Mixinng packages with different packagemanagers like pip/apt is a bad idea, better use a venv for packages not provides by your system

    Debian has good warning when running pip as root

    WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead
    Zwischen anonym sein wollen und seine Daten nicht verkaufen wollen, liegen zwei Welten. Wenn man sich einen kostenpflichtigen Dienst sucht, dann meist, weil man für diese Dienstleistung zahlt und nicht selbst das Produkt sein will.


  8. #8
    Registrierter Benutzer
    Registriert seit
    Aug 2017
    Beiträge
    15
    I have inserted a pd.show_versions() form more Informations and it looks good for me. The python-program is running well when i comment out pyarrow, but i have big files and make
    plots from it and the performace with pyarrow is much better. I will got it running without any special enviroment for a project.


    $ find /usr -name "pyarrow*"
    /usr/local/lib/python3.9/dist-packages/pyarrow
    /usr/local/lib/python3.9/dist-packages/pyarrow/src/arrow/python/pyarrow_api.h
    /usr/local/lib/python3.9/dist-packages/pyarrow/src/arrow/python/pyarrow.cc
    /usr/local/lib/python3.9/dist-packages/pyarrow/src/arrow/python/pyarrow.h
    /usr/local/lib/python3.9/dist-packages/pyarrow/src/arrow/python/pyarrow_lib.h
    /usr/local/lib/python3.9/dist-packages/pyarrow/tests/pyarrow_cython_example.pyx
    /usr/local/lib/python3.9/dist-packages/pyarrow/include/arrow/python/pyarrow_api.h
    /usr/local/lib/python3.9/dist-packages/pyarrow/include/arrow/python/pyarrow.h
    /usr/local/lib/python3.9/dist-packages/pyarrow/include/arrow/python/pyarrow_lib.h
    /usr/local/lib/python3.9/dist-packages/pyarrow-14.0.2.dist-info
    $ ./plotwp-test.py

    INSTALLED VERSIONS
    ------------------
    commit : b5958ee1999e9aead1938c0bba2b674378807b3d
    python : 3.9.2.final.0
    python-bits : 64
    OS : Linux
    OS-release : 5.10.0-26-amd64
    Version : #1 SMP Debian 5.10.197-1 (2023-09-29)
    machine : x86_64
    processor :
    byteorder : little
    LC_ALL : de_DE.UTF-8
    LANG : de_DE.UTF-8
    LOCALE : de_DE.UTF-8

    pandas : 1.1.5
    numpy : 1.26.2
    pytz : 2021.1
    dateutil : 2.8.1
    pip : 23.0.1
    setuptools : 52.0.0
    Cython : None
    pytest : 6.0.2
    hypothesis : None
    sphinx : None
    blosc : None
    feather : None
    xlsxwriter : None
    lxml.etree : 4.6.3
    html5lib : None
    pymysql : None
    psycopg2 : None
    jinja2 : None
    IPython : None
    pandas_datareader: None
    bs4 : None
    bottleneck : None
    fsspec : None
    fastparquet : None
    gcsfs : None
    matplotlib : 3.3.4
    numexpr : None
    odfpy : None
    openpyxl : None
    pandas_gbq : None
    pyarrow : 14.0.2
    pytables : None
    pyxlsb : None
    s3fs : None
    scipy : None
    sqlalchemy : None
    tables : None
    tabulate : None
    xarray : None
    xlrd : None
    xlwt : None
    numba : None

    INSTALLED VERSIONS
    ------------------
    commit : b5958ee1999e9aead1938c0bba2b674378807b3d
    python : 3.9.2.final.0
    python-bits : 64
    OS : Linux
    OS-release : 5.10.0-26-amd64
    Version : #1 SMP Debian 5.10.197-1 (2023-09-29)
    machine : x86_64
    processor :
    byteorder : little
    LC_ALL : de_DE.UTF-8
    LANG : de_DE.UTF-8
    LOCALE : de_DE.UTF-8

    pandas : 1.1.5
    numpy : 1.26.2
    pytz : 2021.1
    dateutil : 2.8.1
    pip : 23.0.1
    setuptools : 52.0.0
    Cython : None
    pytest : 6.0.2
    hypothesis : None
    sphinx : None
    blosc : None
    feather : None
    xlsxwriter : None
    lxml.etree : 4.6.3
    html5lib : None
    pymysql : None
    psycopg2 : None
    jinja2 : None
    IPython : None
    pandas_datareader: None
    bs4 : None
    bottleneck : None
    fsspec : None
    fastparquet : None
    gcsfs : None
    matplotlib : 3.3.4
    numexpr : None
    odfpy : None
    openpyxl : None
    pandas_gbq : None
    pyarrow : 14.0.2
    pytables : None
    pyxlsb : None
    s3fs : None
    scipy : None
    sqlalchemy : None
    tables : None
    tabulate : None
    xarray : None
    xlrd : None
    xlwt : None
    numba : None
    Source-Statement with Error:
    df = pd.read_csv(
    CSV_FILENAME,
    #engine="pyarrow",
    header=0,
    sep=";",
    usecols=["DateTime", "Verdichter_laeuft_seit", "Temp_EQ_Eintritt"],
    parse_dates=["DateTime"],
    dtype={"Verdichter_laeuft_seit": "float", "Temp_EQ_Eintritt": "float"}
    )

  9. #9
    Registrierter Benutzer
    Registriert seit
    Apr 2009
    Ort
    Erde
    Beiträge
    2.804
    I suspect that the system is already in a problematic state (precisely because no venvs were used). A quick search suggests that there are multiple installations of PyArrow that are interfering sa https://github.com/pandas-dev/pandas/issues/24976

    To verify this I would either create a clean venv or a fresh system and check if I still run into that issue or not
    Gruß nopes
    (,,,)---(^.^)---(,,,) /var/log/messages | grep cat

Ähnliche Themen

  1. Ubuntu 10.04 fresh-install: File not found kurz vor grub-Menü
    Von smartysmart34 im Forum System installieren und konfigurieren
    Antworten: 0
    Letzter Beitrag: 17.09.10, 08:45
  2. ./install.pl ist sichtbar und lesbar aber trotzdem "not found"
    Von qwerty@email.de im Forum System installieren und konfigurieren
    Antworten: 1
    Letzter Beitrag: 28.11.08, 12:26
  3. Python - Paramiko für python 2.5
    Von Moesli im Forum Anwendungen Allgemein, Software
    Antworten: 2
    Letzter Beitrag: 13.12.07, 11:36
  4. SUCHE: fuse python bindings für python 2.4 und Suse 10
    Von skurial im Forum System installieren und konfigurieren
    Antworten: 4
    Letzter Beitrag: 20.11.05, 17:21
  5. Python command not found
    Von hever im Forum System installieren und konfigurieren
    Antworten: 4
    Letzter Beitrag: 22.07.03, 14:24

Lesezeichen

Berechtigungen

  • Neue Themen erstellen: Nein
  • Themen beantworten: Nein
  • Anhänge hochladen: Nein
  • Beiträge bearbeiten: Nein
  •