X Tutup
The Wayback Machine - https://web.archive.org/web/20241218210530/https://github.com/python/cpython/issues/50548
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pyexpat build failure on Solaris 10 for 2.6.1/2.6.2 #50548

Closed
enchanter mannequin opened this issue Jun 17, 2009 · 8 comments
Closed

pyexpat build failure on Solaris 10 for 2.6.1/2.6.2 #50548

enchanter mannequin opened this issue Jun 17, 2009 · 8 comments
Labels
build The build process and cross-build topic-XML

Comments

@enchanter
Copy link
Mannequin

enchanter mannequin commented Jun 17, 2009

BPO 6299
Nosy @vstinner, @koobs
Files
  • python-issue6299.patch
    		•

    Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
closed_at = None
created_at = <Date 2009-06-17.20:07:52.681>
labels = ['build']
title = 'pyexpat build failure on Solaris 10 for 2.6.1/2.6.2'
updated_at = <Date 2014-02-04.13:14:23.856>
user = 'https://bugs.python.org/enchanter'

    bugs.python.org fields:

    activity = <Date 2014-02-04.13:14:23.856>
actor = 'vstinner'
assignee = 'none'
closed = False
closed_date = None
closer = None
components = ['Build']
creation = <Date 2009-06-17.20:07:52.681>
creator = 'enchanter'
dependencies = []
files = ['33902']
hgrepos = []
issue_num = 6299
keywords = ['patch']
message_count = 7.0
messages = ['89478', '97311', '189732', '210162', '210189', '210201', '210215']
nosy_count = 3.0
nosy_names = ['vstinner', 'enchanter', 'koobs']
pr_nums = []
priority = 'normal'
resolution = None
stage = None
status = 'open'
superseder = None
type = 'compile error'
url = 'https://bugs.python.org/issue6299'
versions = ['Python 2.7', 'Python 3.3', 'Python 3.4']
    @enchanter
    Copy link
    Mannequin Author

    enchanter mannequin commented Jun 17, 2009

    I've built Python 2.6.1 and Python 2.6.2 on x86_64-sun-solaris2.10 using
    the Sun Workshop Express (200903) toolchain. I'm building in 64 bit mode.

    Most stuff builds just fine (even warnings are rare), but pyexpat fails
    to link with this error:

    building 'pyexpat' extension
    creating
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat
    cc -xtarget=native -m64 -xarch=native -KPIC -DNDEBUG -Xa -xO4 -xstrconst
    -mt -xtarget=native -m64 -xarch=native -I/local/include
    -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -DHAVE_EXPAT_CONFIG_H=1 -DUSE_PYEXPAT_CAPI
    -I/local/src/RPM/BUILD/Python-2.6.2/./Modules/expat -I.
    -I/local/src/RPM/BUILD/Python-2.6.2/./Include -I. -IInclude -I./Include
    -I/local/include -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -I/usr/local/include
    -I/local/src/RPM/BUILD/Python-2.6.2/Include
    -I/local/src/RPM/BUILD/Python-2.6.2 -c
    /local/src/RPM/BUILD/Python-2.6.2/Modules/pyexpat.c -o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/pyexpat.o
    "/local/src/RPM/BUILD/Python-2.6.2/Modules/pyexpat.c", line 1574:
    warning: assignment type mismatch:
    pointer to void "=" pointer to function(pointer to void, pointer to
    const ch
    ar, int) returning void
    cc -xtarget=native -m64 -xarch=native -KPIC -DNDEBUG -Xa -xO4 -xstrconst
    -mt -xtarget=native -m64 -xarch=native -I/local/include
    -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -DHAVE_EXPAT_CONFIG_H=1 -DUSE_PYEXPAT_CAPI
    -I/local/src/RPM/BUILD/Python-2.6.2/./Modules/expat -I.
    -I/local/src/RPM/BUILD/Python-2.6.2/./Include -I. -IInclude -I./Include
    -I/local/include -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -I/usr/local/include
    -I/local/src/RPM/BUILD/Python-2.6.2/Include
    -I/local/src/RPM/BUILD/Python-2.6.2 -c
    /local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmlparse.c -o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmlparse.o
    cc -xtarget=native -m64 -xarch=native -KPIC -DNDEBUG -Xa -xO4 -xstrconst
    -mt -xtarget=native -m64 -xarch=native -I/local/include
    -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -DHAVE_EXPAT_CONFIG_H=1 -DUSE_PYEXPAT_CAPI
    -I/local/src/RPM/BUILD/Python-2.6.2/./Modules/expat -I.
    -I/local/src/RPM/BUILD/Python-2.6.2/./Include -I. -IInclude -I./Include
    -I/local/include -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -I/usr/local/include
    -I/local/src/RPM/BUILD/Python-2.6.2/Include
    -I/local/src/RPM/BUILD/Python-2.6.2 -c
    /local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmlrole.c -o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmlrole.o
    cc -xtarget=native -m64 -xarch=native -KPIC -DNDEBUG -Xa -xO4 -xstrconst
    -mt -xtarget=native -m64 -xarch=native -I/local/include
    -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include -DHAVE_EXPAT_CONFIG_H=1 -DUSE_PYEXPAT_CAPI
    -I/local/src/RPM/BUILD/Python-2.6.2/./Modules/expat -I.
    -I/local/src/RPM/BUILD/Python-2.6.2/./Include -I. -IInclude -I./Include
    -I/local/include -I/local/
    gnu/include -I/local/BerkeleyDB/include -I/local/openssl/include
    -I/usr/local/include -I/local/src/RPM/BUILD/Python-2.6.2/Include
    -I/local/src/RPM/BUILD/Python-2.6.2 -c
    /local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmltok.c -o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmltok.o
    cc -xtarget=native -m64 -xarch=native -G -L/local/lib/64
    -L/local/gnu/lib/64 -L/local/BerkeleyDB/lib/64 -L/local/openssl/lib/64
    -L/local/lib/64 -L/local/gnu/lib/64 -L/local/BerkeleyDB/lib/64
    -L/local/openssl/lib/64 -DNDEBUG -Xa -xO4 -xstrconst -mt -xtarget=native
    -m64 -xarch=native -I/local/include -I/local/gnu/include
    -I/local/BerkeleyDB/include -I/local/openssl/include -I. -IInclude
    -I./Include
    -I/local/include -I/local/gnu/include -I/local/BerkeleyDB/include
    -I/local/openssl/include
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/pyexpat.o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmlparse.o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmlrole.o
    build/temp.solaris-2.10-i86pc-2.6/local/src/RPM/BUILD/Python-2.6.2/Modules/expat/xmltok.o
    -L/local/lib -L/local/lib/64 -L/local/gnu/lib/64
    -L/local/BerkeleyDB/lib/64 -L/local/openssl/lib/64 -L/usr/local/lib -o
    build/lib.solaris-2.10-i86pc-2.6/pyexpat.so
    *** WARNING: renaming "pyexpat" since importing it failed: ld.so.1:
    python: fatal: relocation error: file
    build/lib.solaris-2.10-i86pc-2.6/pyexpat.so: symbol
    XML_SetCharacterDataHandler: referenced symbol not found

    I never tried any of the 2.5.x Python builds, but I can tell you that I
    built several versions in the 2.4.x series, and none of those versions
    had problems compiling or linking pyexpat.

    I do have expat 2.0.1 (also built with the Sun compiler in 64 bit mode)
    installed on the system, but from the comments in setup.py and the
    compiler output, it appears clear that the expat components that ship
    with Python 2.6.x are the ones that are being compiled and attempted for
    use in the link.

    Please let me know if I can provide any relevant information.

    Thanks,

    Tim (the Enchanter)

    @enchanter enchanter mannequin added build The build process and cross-build labels Jun 17, 2009
    @enchanter
    Copy link
    Mannequin Author

    enchanter mannequin commented Jan 6, 2010

    This still happens in 2.6.3 and 2.6.4 so I spent some time looking at it.

    It's happening because of a combination of issues, but ultimately it's because python's build isn't making certain that it's including its private copy of expat.h and expat_external.h.

    Basically, this

    -DHAVE_EXPAT_CONFIG_H=1 -DUSE_PYEXPAT_CAPI -I/local/src/RPM/BUILD/Python-2.6.4/./Modules/expat

    should *precede* CFLAGS, rather than coming after it.

    To reproduce:

    • install expat 2.0.1 under /usr/local or /opt (or any prefix other
      than just /usr)
    • include -I$prefix/include in CFLAGS when configuring Python
    • make

    This results in the special includes for python's local copy of the expat sources being *after* the -I/usr/local/include on the command line, so the copy of expat.h and expat_external.h in /usr/local/include are found first, and that causes the link failure when generating pyexpat.so.

    Note that it's not just as simple as not including -I/usr/local/include in CFLAGS, because there may be other locally-installed packages that python should be finding to build other modules.

    The real fix is for Python's build machinery to make certain that the special includes for Python's private expat sources precede CFLAGS.

    @BreamoreBoy
    Copy link
    Mannequin

    BreamoreBoy mannequin commented May 21, 2013

    Is this still a problem given that both Python and Solaris have moved on?

    @enchanter
    Copy link
    Mannequin Author

    enchanter mannequin commented Feb 3, 2014

    I just tried Python 2.7.6 on x86_64-pc-openindiana151a9. The core issue is still present. It's not currently causing a build failure, but only because OpenIndiana 151a9 and Python are using the exact same version of Expat, so the headers that python's build process is mistakenly including from the operating system match the code that it's compiling internally.

    My comment in msg97311 on 2010-01-06 is still valid. If Python's build process is going to use internal expat, then it better ensure that the compilation flags it needs for that (including the -I) comes before the standard CFLAGS & CPPFLAGS.

    @koobs
    Copy link

    koobs commented Feb 4, 2014

    The issue applies and is reproducible for all versions 2.6 through 3.5.

    This is the changeset we applied to all FreeBSD Ports to fix the issue:

    https://svnweb.freebsd.org/ports?view=revision&revision=326729

    One specific example (Python 3.3):

    1. Use CPPFLAGS over CFLAGS (See before *and* after comments):

    https://svnweb.freebsd.org/ports/head/lang/python33/Makefile?r1=326729&r2=326728&pathrev=326729

    1. Revert the absolutely crazy complexity in Makefile.pre.in by stripping out CONFIGURE_*, allowing once again
      ./configure to substitute the right variables based on what has been passed to it:

    https://svnweb.freebsd.org/ports/head/lang/python33/files/patch-Makefile.pre.in?r1=326729&r2=326728&pathrev=326729

    This results in:

    Makefile.pre.in:
    CPPFLAGS= @CPPFLAGS@ <-- YAY!
    PY_CPPFLAGS= $(BASECPPFLAGS) -I. -IInclude -I$(srcdir)/Include $(CPPFLAGS) <-- YAY

    Makefile:
    CPPFLAGS= -I/usr/local/include - YAY!
    PY_CPPFLAGS= $(BASECPPFLAGS) -I. -IInclude -I$(srcdir)/Include $(CPPFLAGS) <-- YAY!

    The root cause *requiring* the use of CPPFLAGS, is PY_CFLAGS before PY_CPPFLAGS here:

    PY_CORE_CFLAGS= $(PY_CFLAGS) $(PY_CPPFLAGS) $(CFLAGSFORSHARED) -DPy_BUILD_CORE

    As per https://www.gnu.org/prep/standards/html_node/Command-Variables.html

    "Put CFLAGS last in the compilation command, after other variables containing compiler options, so the user can use CFLAGS to override the others."

    We must use CPPFLAGS, because CFLAGS has been broken in one way or another for a long time. The target state is *both* must just work.

    This can only happen if the standard user-serviceable autoconf and Makefile variables are left alone, not extended or overridden, and behave in exactly the same way whether provided in the environment for *either* ./configure, make, or *both*.

    For the most recent chapter in the C[PP|LD]FLAGS/Makefile book, see: 92a9dc668c95 from bpo-9189 which was sound in intent, but in execution left us deeper down the rabbit-hole.

    Moving forward and as a first step, what does everyone think of switching the order of
    $(PY_CFLAGS) $(PY_CPPFLAGS) in PY_CORE_CFLAGS= ?

    I hope (but I'm not holding my breath) that nothing relies on the current ordering.

    @koobs
    Copy link

    koobs commented Feb 4, 2014

    Setting versions to correctly reflect those affected.

    @koobs
    Copy link

    koobs commented Feb 4, 2014

    Adding a patch for reference only

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    @kumaraditya303
    Copy link
    Contributor

    Closing as Python versions 2.6 - 3.5 are not supported. If this issue still exists, create a new issue targeting supported python versions with a reproducer.

    @kumaraditya303 kumaraditya303 closed this as not planned Won't fix, can't repro, duplicate, stale Jun 9, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Assignees
    No one assigned
    Labels
    build The build process and cross-build topic-XML
    Projects
    None yet
    Milestone
    No milestone
    Development

    No branches or pull requests
    3 participants
    @koobs @serhiy-storchaka @kumaraditya303
    X Tutup