[Question] using C headers/libraries with rootcling/CMake interface

[txjmb]

We have some C shared libraries that we are wanting to package with cppyy for use in Python. We are able to use cppyy.c_include, or manually add the extern "C" code to the header files and can import that way well. However, when we use the CMake interface, things don't seem to work. After looking into the issue, it seems as if rootcling does not parse functions in C headers when an #ifdef __cplusplus extern "C" { #endif conditional block is wrapped around the header code. If we remove the conditional block from the header, the functions are mapped into the .map file and presumably into the dictionary, however the header file seems to be C++ name mangled and linking doesn't work, similar to if we tried to use a C library from cppyy with a header file without using the c_include function. Not surprising, however it makes it seem like it may not be possible to use the CMake packaging for C projects. Is there some way to tell rootcling to read the header file as a C header?

I'm not sure there is a way around the packaging side of this without wrapping everything in C++, but wanted to see if you had any ideas.

Thank you for your time and for cppyy!

[wlav]

The dictionary doesn't actually do anything for functions, so it shouldn't matter, as long as the header itself is listed in the dictionary. (W/o the I/O part, the packaging of headers and paths is the only relevant use of dictionaries at this point. Headers are still only loaded with #include as containing a copy in-text does not compile on MS WIndows (limit of 2048 chars in a string, IIRC.)

c_include does nothing more than sandwich the actual #include within __cplusplus extern "C".

I'll try to reproduce. (rootcling is a mess and the cmake fragments were contributed, so may take me some time, but I suspect it's something simple in how rootcling retrieves the decl context.)

[txjmb]

As always, thank you for your help. I did look into the c_include code and saw the extern "C" wrapping, which makes sense. I think we've narrowed down the behavior to rootcling's creation of the .map file when there are ifdefs. If we do not have the ifdefs, then the function shows up in the .map file and with a (modified) version of the python package template initializor.py, it gets added to our python namespace properly so we can simply use import helloworld as hw; hw.hello() instead of cppy.gbl.hello(). It is added but because it didn't have the extern "C" ifdefs, it is name mangled, so when called we get the expected name-mangling-induced error:

IncrementalExecutor::executeFunction: symbol '_Z5hellov' unresolved while linking symbol '__cf_4'! You are probably missing the definition of hello() Maybe you need to load the corresponding shared library?

So, it seems like ifdefs themselves are the "root" of the issue for the creation of the .map files. If rootcling could just ignore them when parsing the header files to create the .map file, it would work. Not sure why it is affected by them at all.

We're using some "hello world" examples for our testing, so should be able to repro:

main.c:

#include<stdio.h>
#include "main.h"

const char* hello()
{
    printf("Hello World");

    char *name = "Hello World!!!!";
    return name;
}

main.h:

#ifdef __cplusplus
extern "C" {
#endif

const char* hello();

#ifdef __cplusplus
 }
#endif

btw, here is our change to the initializor.py (added 'function' to the list to match what is created in .map file (when there are no ifdefs, of course):

    #
    # Iterate over all the items at the top level of each file, and add them
    # to the pkg.
    #
    for file in files:
        for child in file["children"]:
            if not child["kind"] in ('class', 'var', 'namespace', 'typedef', 'function'):
                continue
            simplenames = child["name"].split('::')
            add_to_pkg(file["name"], child["kind"], simplenames, child)

CMAKE snippet:

cppyy_add_bindings(
   "HELLOWORLD" "${PROJECT_VERSION}" "****" "****@***.***"
   LANGUAGE_STANDARD "17"
   COMPILE_OPTIONS ${COMPILER_OPTIONS}
   INCLUDE_DIRS ${INCLUDE_DIRS}
   LINK_LIBRARIES HelloWorld
   H_DIRS ${H_DIRS}
   H_FILES ${H_FILES}
)

[wlav]

Thanks, so I misunderstood the question: this is the .map file, not the .rootmap file. I.e. this is not rootcling, but libclang, so scratch what I said above. :}

Turns out that the libclang does not support extern "C" functions and the function ends up being labeled an UNEXPOSED_DECL, with AFAICT, no further information available. I can try to figure out a workaround, since per Clang's docs:

# Unexposed declarations have the same operations as any other kind of
# declaration; one can extract their location information, spelling, find their
# definitions, etc. However, the specific kind of the declaration is not
# reported.

so in theory it can be decomposed, but it's not already done (which is what is the case for recognized functions).

[txjmb]

Thank you for digging into this. Yes, this is the .map file. I made an incorrect assumption it was rootcling. Seems like any workaround would be a helpful improvement in integrating C. Let me know if you need any more information, testing, etc.

[wlav]

The .map files are a bit of a niche use case at this point. The idea was to force generation of all proxies for easy use with tab-completion in IDEs, as well as to have dir() function properly. Today, dir() has greatly improved and stub files are probably a better approach to work with IDEs (yes, doesn't exist as-is, more work :) ). Furthermore, generating all proxies on load rather than lazily would negate the benefit of pre-compiled modules (which contrary to the pre-compiled header are not fully deserialized on load and should help reduce memory overhead).

What bothers me more is that extern "C" functions do not show up in dir() either.