X Tutup
Skip to content

[CI/CD] Azure Pipeline: issue with _sqlite3 module #120159

@picnixz

Description

@picnixz

Bug report

Bug description:

This is something I wanted to point out but I'm not sure whether it's a known bug or not (if so, feel free to close the issue). I couldn't find a related issue.


The Azure Pipeline is failing on Windows, similarly to #84395 but with a more explicit error. There are two failing tests, namely

  • test.test_ctypes.test_loading.LoaderTest.test_load_dll_with_flags
  • test.test_import.ImportTests.test_dll_dependency_import

Both of them fail because they cannot find the DLL related to the sqlite3 module. The first failure failure is actually very clear:

 Traceback (most recent call last):
  File "D:\a\1\b\layout-appx-win32\Lib\test\test_ctypes\test_loading.py", line 157, in test_load_dll_with_flags
    shutil.copy(os.path.join(os.path.dirname(src), "sqlite3" + ext),
    ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
                os.path.join(tmp, "sqlite3" + ext))
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\a\1\b\layout-appx-win32\Lib\shutil.py", line 428, in copy
    copyfile(src, dst, follow_symlinks=follow_symlinks)
    ~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\a\1\b\layout-appx-win32\Lib\shutil.py", line 260, in copyfile
    with open(src, 'rb') as fsrc:
         ~~~~^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\a\\1\\b\\layout-appx-win32\\DLLs\\sqlite3.dll'

I think the issue is with the DLLs directory which is never created (AFAICT), as illustrated by the current code:

with os_helper.temp_dir() as tmp:
# We copy two files and load _sqlite3.dll (formerly .pyd),
# which has a dependency on sqlite3.dll. Then we test
# loading it in subprocesses to avoid it starting in memory
# for each test.
target = os.path.join(tmp, "_sqlite3.dll")
shutil.copy(src, target)
shutil.copy(os.path.join(os.path.dirname(src), "sqlite3" + ext),
os.path.join(tmp, "sqlite3" + ext))

There should be some

tmp2 = os.path.join(tmp, "DLLs")
os.mkdir(tmp2)

similar to what is being done in test.test_import.ImportTests.test_dll_dependency_import. Note that the source DLL should be present since it is generated during the build.

The second failure occurs with:

ERROR: test_dll_dependency_import (test.test_import.ImportTests.test_dll_dependency_import)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "D:\a\1\b\layout-appx-win32\Lib\test\test_import\__init__.py", line 762, in test_dll_dependency_import
...
subprocess.CalledProcessError: Command '[
	'C:\\Users\\VssAdministrator\\AppData\\Local\\Temp\\test_python_w9okac52\\tmp7ps_2s1g\\python.exe', 
	'-Sc', 
	"import os;p = os.add_dll_directory('D:\\\\a\\\\1\\\\b\\\\layout-appx-win32\\\\DLLs');import _sqlite3;p.close"
]' returned non-zero exit status 1.

Here I think what occurs is that the files are not copied correctly (the DLLs directory is correctly created but I assume that the issue is due to the missing dll file, or perhaps it's moved incorrectly). Here is the code setting up the environment:

from _winapi import GetModuleFileName
dllname = GetModuleFileName(sys.dllhandle)
pydname = importlib.util.find_spec("_sqlite3").origin
depname = os.path.join(
os.path.dirname(pydname),
"sqlite3{}.dll".format("_d" if "_d" in pydname else ""))
with os_helper.temp_dir() as tmp:
tmp2 = os.path.join(tmp, "DLLs")
os.mkdir(tmp2)
pyexe = os.path.join(tmp, os.path.basename(sys.executable))
shutil.copy(sys.executable, pyexe)
shutil.copy(dllname, tmp)
for f in glob.glob(os.path.join(glob.escape(sys.prefix), "vcruntime*.dll")):
shutil.copy(f, tmp)
shutil.copy(pydname, tmp2)

CPython versions tested on:

CPython main branch

Operating systems tested on:

Other

Metadata

Metadata

Assignees

No one assigned

    Labels

    OS-windowstype-bugAn unexpected behavior, bug, or error

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      X Tutup