@@ -15,7 +15,7 @@ The CppInterop comes with using it is a dynamic shared library,
1515 libInterop = ctypes.CDLL(" ./libclangCppInterOp.so" )
1616
1717 The above method of usage is for Python; for C, we can include the headers of
18- the library. Including this library in our programme enables the user to use
18+ the library. Including this library in our program enables the user to use
1919the abilities of CppInterOp. CppInterOp helps programmers with multiple
2020verifications such as isClass, isBoolean, isStruct, and many more in different
2121languages. With the interop layer, we can access the scopes, namespaces of
@@ -28,10 +28,10 @@ Using LLVM as external library
2828
2929In CppInterOp, we are leveraging Clang as a library for interoperability purposes.
3030To use Clang, we need to pass the Clang configuration to the CMake build system,
31- so that the build system recognises the configuration and enables usage of Clang
31+ so that the build system recognizes the configuration and enables usage of Clang
3232and LLVM.
3333We can consider clang-repl as a state manager, where CppInterOp allows you to
34- query the state from the state manager. Thereafter, Cppyy uses this to create
34+ query the state from the state manager. Thereafter, cppyy uses this to create
3535Python objects for C++.
3636
3737.. code-block :: bash
@@ -65,35 +65,258 @@ robust (simple function calls, no inheritance, etc.). The goal is to make it as
6565close to the compiler API as possible, and each routine should do just one thing.
6666that it was designed for.
6767
68- Further Enhancing the Dynamic/Automatic bindings in CPPYY
69- =========================================================
70- The main use case for CppInterOp is the CPPYY service. CPPYY is an
71- automatic run-time bindings generator for Python and C++, and supports a wide
72- range of C++ features (e.g., template instantiation). It operates on demand and
73- generates only what is necessary. It requires a compiler (Cling or Clang-REPL).
74- that can be available during programme runtime.
68+ How cppyy leverages CppInterOp
69+ ===============================
70+
71+ cppyy is a run-time Python-C++ bindings generator for calling C++ from Python
72+ and Python from C++. Interestingly, it uses C++ interactively by using the
73+ compiler as a service. This is made possible by the CppInterOp library.
74+ Following are some of the ways cppyy leverages CppInterOp for better
75+ performance and usability.
76+
77+ 1. **CppInterOp enables interoperability with C++ code **: CppInterOp provides a
78+ minimalist and robust interface for language interoperability on the fly,
79+ which helps CPPYY generate dynamic Python-C++ bindings by using a C++
80+ interpreter (e.g., Clang-REPL/ Cling) and LLVM.
81+
82+ 2. **Reducing dependencies **: Reducing domain-specific dependencies of cppyy
83+ (e.g., on the Cling interpreter and the ROOT framework) to enable more
84+ generalized usage.
85+
86+ 3. **LLVM Integration **: CppInterOp is designed to be used as a part of the
87+ LLVM toolchain (as part of Clang-REPL) that can then be used as a runtime
88+ compiler for CPPYY. This simplifies the codebase of CPPYY and enhances its
89+ performance.
90+
91+ 4. **Making C++ More Social **: CppInterOp and cppyy help data scientists that
92+ are working with legacy C++ code experiment with simpler, more interactive
93+ languages, while also interacting with larger communities.
94+
95+ **CppInterOp enables interoperability with C++ code **
96+
97+ cppyy is a major use case for CppInterOp. cppyy is an automatic run-time
98+ bindings generator for Python and C++, and supports a wide range of C++
99+ features (e.g., template instantiation). It operates on demand and generates
100+ only what is necessary. It requires a compiler (Cling or Clang-REPL) that can
101+ be available during program runtime.
102+
103+ **Reducing Dependencies **
104+
105+ Recent work done on cppyy has been focused on reducing dependencies on
106+ domain-specific infrastructure (e.g., the ROOT framework). Using an independent
107+ library such as CppInterOp helps accomplish that, while also reducing the code
108+ bloat in cppyy.
109+
110+ The CppInterOp library can be configured to use the newly developed Clang-Repl
111+ backend available in LLVM upstream (or to use the Cling legacy backend, for
112+ compatibility with High Energy Physics applications).
113+
114+ Only a small set of APIs are needed to connect to the interpreter (Clang-Repl/
115+ Cling), since other APIs are already available in the standard compiler. This
116+ is one of the reasons that led to the creation of CppInterOp (a library of
117+ helper functions), that can help extract out things that are unnecessary for
118+ for core cppyy functionality.
119+
120+ The cppyy API surface is now incomparably smaller and simpler than what it used
121+ to be.
122+
123+ **LLVM Integration **
75124
76125Once CppInterOp is integrated with LLVM's Clang-REPL component (that can then
77- be used as a runtime compiler for CPPYY ), it will further enhance CPPYY 's
126+ be used as a runtime compiler for cppyy ), it will further enhance cppyy 's
78127performance in the following ways:
79128
80129
81- **Simpler codebase: ** The removal of string parsing logic will lead to a simpler
82- code base.
130+ - *Simpler codebase: * The removal of string parsing logic will lead to a
131+ simpler code base.
132+
133+ - *Built into the LLVM toolchain: * The CppInterOp interfaces will be part of
134+ the LLVM toolchain (as part of Clang-REPL).
135+
136+ - *Better C++ Support: * Finer-grained control over template instantiation is
137+ available through CppInterOp.
138+
139+ - *Fewer Lines of Code: * A lot of dependencies and workarounds will be
140+ removed, reducing the lines of code required to execute cppyy.
141+
142+ - *Well tested interoperability Layer: * The CppInterOp interfaces have full
143+ unit test coverage.
144+
145+ **Making C++ More Social **
146+
147+ cppyy is the first use case demonstrating how CppInterOp can enable C++ to be
148+ more easily interoperable with other languages. This helps many data scientists
149+ that are working with legacy C++ code and would like to use simpler, more
150+ interactive languages.
151+
152+ The goal of these enhancements is to eventually land these interoperability
153+ tools (including CppInterOp) to greater communities like LLVM and Clang, to
154+ enable C++ to interact with other languages besides Python.
155+
156+ Example: Template Instantiation
157+ -------------------------------
158+
159+ The developmental cppyy version can run basic examples such as the one
160+ here. Features such as standalone functions and basic classes are also
161+ supported.
162+
163+ C++ code (Tmpl.h)
164+
165+ ::
166+
167+ template <typename T>
168+ struct Tmpl {
169+ T m_num;
170+ T add (T n) {
171+ return m_num + n;
172+ }
173+ };
174+
175+ Python Interpreter
176+
177+ ::
178+
179+ >>> import cppyy
180+ >>> import cppyy.gbl as Cpp
181+ >>> cppyy.include("Tmpl.h")
182+ >>> tmpl = Tmpl[int]()
183+ >>> tmpl.m_num = 4
184+ >>> print(tmpl.add(5))
185+ 9
186+ >>> tmpl = Tmpl[float]()
187+ >>> tmpl.m_num = 3.0
188+ >>> print(tmpl.add(4.0))
189+ 7.0
190+
191+ Where does the cppyy code reside?
192+ ---------------------------------
193+
194+ Following are the main components where cppyy logic (with Compiler Research
195+ Organization’s customizations started by `sudo-panda `_) resides:
196+
197+ - `cppyy <https://github.com/compiler-research/cppyy >`_
198+ - `cppyy-backend <https://github.com/compiler-research/cppyy-backend >`_
199+ - `CPyCppyy <https://github.com/compiler-research/CPyCppyy >`_
200+
201+ ..
202+
203+ Note: These are forks of the `upstream cppyy `_ repos created by `wlav `_.
204+
205+ CppInterOp is a separate library that helps these packages communicate with C++
206+ code.
207+
208+ - `CppInterOp <https://github.com/compiler-research/CppInterOp/tree/main >`_
83209
84- ** LLVM Integration: ** The CppInterOp interfaces will be part of the LLVM
85- toolchain (as part of Clang-REPL).
210+ How cppyy components interact with each other
211+ ---------------------------------------------
86212
87- **Better C++ Support: ** C++ features such as Partial Template Specialisation will
88- be available through CppInterOp.
213+ cppyy is made up of the following packages:
89214
90- **Fewer Lines of Code: ** A lot of dependencies and workarounds will be removed,
91- reducing the lines of code required to execute CPPYY.
215+ - A frontend: cppyy,
92216
93- **Well tested interoperability Layer: ** The CppInterOp interfaces have full
94- unit test coverage.
217+ - A backend: cppyy-backend, and
95218
219+ - An extension: CPyCppyy.
220+
221+ Besides these, the ``CppInterOp `` library serves as an additional layer on top
222+ of Cling/Clang-REPL that helps these packages in communicating with C++ code.
223+
224+ **1. cppyy-backend **
225+
226+ The `cppyy-backend `_ package forms a layer over ``cppyy ``, for example,
227+ modifying some functionality to provide the functions required for
228+ ``CPyCppyy ``.
229+
230+ `CPyCppyy `_ is a CPython extension module built on top of the same backend
231+ API as PyPy/_cppyy. It thus requires the installation of the cppyy-backend
232+ for use, which will pull in Cling.
233+
234+ ``cppyy-backend `` also adds some `utilities `_ to help with repackaging and
235+ redistribution.
236+
237+ For example, ``cppyy-backend `` initializes the interpreter (using the
238+ ``clingwrapper::ApplicationStarter `` function), adds the required ``include ``
239+ paths, and adds the headers required for cppyy to work. It also adds some
240+ checks and combines two or more functions to help CPyCppyy work.
241+
242+ These changes help ensure that any change in ``cppyy `` doesn’t directly
243+ affect ``CPyCppyy ``, and the API for ``CPyCppyy `` remains unchanged.
244+
245+ **2. CPyCppyy **
246+
247+ The ``CPyCppyy `` package uses the functionality provided by ``cppyy-backend ``
248+ and provides Python objects for C++ entities. ``CPyCppyy `` uses separate proxy
249+ classes for each type of object. It also includes helper classes, for example,
250+ ``Converters.cxx `` helps convert Python type objects to C++ type objects, while
251+ ``Executors.cxx `` is used to execute a function and convert its return value to
252+ a Python object, so that it can be used inside Python.
253+
254+ **3. cppyy **
255+
256+ The cppyy package provides the front-end for Python. It is `included in code `_
257+ (using ``import cppyy ``) to import cppyy in Python. It initializes things on
258+ the backend side, provides helper functions (e.g., ``cppdef() ``, ``cppexec() ``,
259+ etc.) that the user can utilize, and it calls the relevant backend functions
260+ required to initialize cppyy.
261+
262+
263+ Further Reading
264+ ---------------
265+
266+ - `High-performance Python-C++ bindings with PyPy and
267+ Cling <http://cern.ch/wlav/Cppyy_LavrijsenDutta_PyHPC16.pdf> `_
268+
269+ - `Efficient and Accurate Automatic Python Bindings with cppyy &
270+ Cling <https://arxiv.org/abs/2304.02712> `_
271+
272+ - cppyy documentation:
273+ `cppyy.readthedocs.io <http://cppyy.readthedocs.io/ >`_.
274+
275+ - Notebook-based tutorial: `Cppyy
276+ Tutorial <https://github.com/wlav/cppyy/blob/master/doc/tutorial/CppyyTutorial.ipynb> `_.
277+
278+ - `C++ Language Interoperability
279+ Layer <https://compiler-research.org/libinterop/> `_
280+
281+ **Credits: **
282+
283+ - `Wim Lavrijsen <https://github.com/wlav >`_ (Lawrence Berkeley National Lab.)
284+ for his original work in cppyy and mentorship towards student contributors.
285+
286+ - `Vassil Vasilev <https://github.com/vgvassilev >`_ (Princeton University)
287+ for his mentorship towards Compiler Research Org's student contributors.
288+
289+ - `Baidyanath Kundu <https://github.com/sudo-panda >`_ (Princeton University)
290+ for his research work on cppyy and Numba with `Compiler Research Organization `_
291+ (as discussed in this document).
292+
293+ - `Aaron Jomy <https://github.com/maximusron >`_ (Princeton University) for
294+ continuing this research work with `Compiler Research Organization `_.
96295
97296In case you haven't already installed CppInterop, please do so before proceeding
98297with the Installation And Usage Guide.
99- :doc: `Installation and usage <InstallationAndUsage >`
298+ :doc: `Installation and usage <InstallationAndUsage >`
299+
300+ .. _Compiler Research Organization : https://compiler-research.org/
301+
302+ .. _upstream cppyy : https://github.com/wlav/cppyy
303+
304+ .. _wlav : https://github.com/wlav
305+
306+ .. _utilities : https://cppyy.readthedocs.io/en/latest/utilities.html
307+
308+ .. _included in code : https://cppyy.readthedocs.io/en/latest/starting.html
309+
310+ .. _sudo-panda : https://github.com/sudo-panda
311+
312+ .. _cppyy : https://cppyy.readthedocs.io/en/latest/index.html
313+
314+ .. _CppInterOp : https://github.com/compiler-research/CppInterOp
315+
316+ .. _ROOT meta : https://github.com/root-project/root/tree/master/core/meta
317+
318+ .. _enhancements in cppyy : https://arxiv.org/abs/2304.02712
319+
320+ .. _CPyCppyy : https://github.com/wlav/CPyCppyy
321+
322+ .. _cppyy-backend : https://github.com/wlav/cppyy-backend
0 commit comments