## 問題
You want to use Cython to make a Python extension module that wraps around anexisting C library.
## 解決方案
Making an extension module with Cython looks somewhat similar to writing a hand‐written extension, in that you will be creating a collection of wrapper functions. How‐ever, unlike previous recipes, you won’t be doing this in C—the code will look a lot morelike Python.As preliminaries, assume that the sample code shown in the introduction to this chapterhas been compiled into a C library called libsample. Start by creating a file namedcsample.pxd that looks like this:
# csample.pxd## Declarations of “external” C functions and structures
cdef extern from “sample.h”:
int gcd(int, int)bint in_mandel(double, double, int)int divide(int, int, int [*](#))double avg(double [*](#), int) nogil
ctypedef struct Point:double xdouble y
double distance(Point [*](#), Point [*](#))
This file serves the same purpose in Cython as a C header file. The initial declarationcdef extern from “sample.h” declares the required C header file. Declarationsthat follow are taken from that header. The name of this file is csample.pxd, not sam‐ple.pxd—this is important.Next, create a file named sample.pyx. This file will define wrappers that bridge thePython interpreter to the underlying C code declared in the csample.pxd file:
# sample.pyx
# Import the low-level C declarationscimport csample
# Import some functionality from Python and the C stdlibfrom cpython.pycapsule cimport *
from libc.stdlib cimport malloc, free
# Wrappersdef gcd(unsigned int x, unsigned int y):
> return csample.gcd(x, y)
def in_mandel(x, y, unsigned int n):return csample.in_mandel(x, y, n)def divide(x, y):cdef int remquot = csample.divide(x, y, &rem)return quot, remdef avg(double[:] a):cdef:int szdouble result
sz = a.sizewith nogil:
> result = csample.avg(<double [*](#)> &a[0], sz)
return result
# Destructor for cleaning up Point objectscdef del_Point(object obj):
> pt = <csample.Point [*](#)> PyCapsule_GetPointer(obj,”Point”)free(<void [*](#)> pt)
# Create a Point object and return as a capsuledef Point(double x,double y):
> > cdef csample.Point [*](#)pp = <csample.Point [*](#)> malloc(sizeof(csample.Point))if p == NULL:
> > raise MemoryError(“No memory to make a Point”)
> p.x = xp.y = yreturn PyCapsule_New(<void [*](#)>p,”Point”,<PyCapsule_Destructor>del_Point)
def distance(p1, p2):pt1 = <csample.Point [*](#)> PyCapsule_GetPointer(p1,”Point”)pt2 = <csample.Point [*](#)> PyCapsule_GetPointer(p2,”Point”)return csample.distance(pt1,pt2)
Various details of this file will be covered further in the discussion section. Finally, tobuild the extension module, create a setup.py file that looks like this:
from distutils.core import setupfrom distutils.extension import Extensionfrom Cython.Distutils import build_ext
ext_modules = [
Extension(‘sample',
> [‘sample.pyx'],libraries=[‘sample'],library_dirs=[‘.'])]
setup(name = ‘Sample extension module',cmdclass = {‘build_ext': build_ext},ext_modules = ext_modules
)
To build the resulting module for experimentation, type this:
bash % python3 setup.py build_ext –inplacerunning build_extcythoning sample.pyx to sample.cbuilding ‘sample' extensiongcc -fno-strict-aliasing -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
> -I/usr/local/include/python3.3m -c sample.c-o build/temp.macosx-10.6-x86_64-3.3/sample.o
gcc -bundle -undefined dynamic_lookup build/temp.macosx-10.6-x86_64-3.3/sample.o-L. -lsample -o sample.so
bash %
If it works, you should have an extension module sample.so that can be used as shownin the following example:
>>> import sample
>>> sample.gcd(42,10)
2
>>> sample.in_mandel(1,1,400)
False
>>> sample.in_mandel(0,0,400)
True
>>> sample.divide(42,10)
(4, 2)
>>> import array
>>> a = array.array('d',[1,2,3])
>>> sample.avg(a)
2.0
>>> p1 = sample.Point(2,3)
>>> p2 = sample.Point(4,5)
>>> p1
<capsule object "Point" at 0x1005d1e70>
>>> p2
<capsule object "Point" at 0x1005d1ea0>
>>> sample.distance(p1,p2)
2.8284271247461903
>>>
## 討論
This recipe incorporates a number of advanced features discussed in prior recipes, in‐cluding manipulation of arrays, wrapping opaque pointers, and releasing the GIL. Eachof these parts will be discussed in turn, but it may help to review earlier recipes first.At a high level, using Cython is modeled after C. The .pxd files merely contain C defi‐nitions (similar to .h files) and the .pyx files contain implementation (similar to a .c file).The cimport statement is used by Cython to import definitions from a .pxd file. This isdifferent than using a normal Python import statement, which would load a regularPython module.Although .pxd files contain definitions, they are not used for the purpose of automati‐cally creating extension code. Thus, you still have to write simple wrapper functions.For example, even though the csample.pxd file declares int gcd(int, int) as a func‐tion, you still have to write a small wrapper for it in sample.pyx. For instance:
cimport csample
def gcd(unsigned int x, unsigned int y):return csample.gcd(x,y)
For simple functions, you don’t have to do too much. Cython will generate wrapper codethat properly converts the arguments and return value. The C data types attached to thearguments are optional. However, if you include them, you get additional error checkingfor free. For example, if someone calls this function with negative values, an exceptionis generated:
>>> sample.gcd(-10,2)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "sample.pyx", line 7, in sample.gcd (sample.c:1284)
def gcd(unsigned int x,unsigned int y):
OverflowError: can't convert negative value to unsigned int
>>>
If you want to add additional checking to the wrapper, just use additional wrapper code.For example:
def gcd(unsigned int x, unsigned int y):if x <= 0:raise ValueError(“x must be > 0”)if y <= 0:raise ValueError(“y must be > 0”)
return csample.gcd(x,y)
The declaration of in_mandel() in the csample.pxd file has an interesting, but subtledefinition. In that file, the function is declared as returning a bint instead of an int.This causes the function to create a proper Boolean value from the result instead of asimple integer. So, a return value of 0 gets mapped to False and 1 to True.
Within the Cython wrappers, you have the option of declaring C data types in additionto using all of the usual Python objects. The wrapper for divide() shows an exampleof this as well as how to handle a pointer argument.
def divide(x,y):cdef int remquot = csample.divide(x,y,&rem)return quot, rem
Here, the rem variable is explicitly declared as a C int variable. When passed to theunderlying divide() function, &rem makes a pointer to it just as in C.The code for the avg() function illustrates some more advanced features of Cython.First the declaration def avg(double[:] a) declares avg() as taking a one-dimensionalmemoryview of double values. The amazing part about this is that the resulting functionwill accept any compatible array object, including those created by libraries such asnumpy. For example:>>> import array>>> a = array.array(‘d',[1,2,3])>>> import numpy>>> b = numpy.array([1., 2., 3.])>>> import sample>>> sample.avg(a)2.0>>> sample.avg(b)2.0>>>
In the wrapper, a.size and &a[0] refer to the number of array items and underlyingpointer, respectively. The syntax <double [*](#)> &a[0] is how you type cast pointers to adifferent type if necessary. This is needed to make sure the C avg() receives a pointerof the correct type. Refer to the next recipe for some more advanced usage of Cythonmemoryviews.In addition to working with general arrays, the avg() example also shows how to workwith the global interpreter lock. The statement with nogil: declares a block of code asexecuting without the GIL. Inside this block, it is illegal to work with any kind of normalPython object—only objects and functions declared as cdef can be used. In addition tothat, external functions must explicitly declare that they can execute without the GIL.Thus, in the csample.pxd file, the avg() is declared as double avg(double [*](#), int)nogil.The handling of the Point structure presents a special challenge. As shown, this recipetreats Point objects as opaque pointers using capsule objects, as described inRecipe 15.4. However, to do this, the underlying Cython code is a bit more complicated.First, the following imports are being used to bring in definitions of functions from theC library and Python C API:
from cpython.pycapsule cimport *from libc.stdlib cimport malloc, free
The function del_Point() and Point() use this functionality to create a capsule objectthat wraps around a Point * pointer. The declaration cdef del_Point() declaresdel_Point() as a function that is only accessible from Cython and not Python. Thus,this function will not be visible to the outside—instead, it’s used as a callback functionto clean up memory allocated by the capsule. Calls to functions such as PyCapsule_New(), PyCapsule_GetPointer() are directly from the Python C API and are usedin the same way.The distance() function has been written to extract pointers from the capsule objectscreated by Point(). One notable thing here is that you simply don’t have to worry aboutexception handling. If a bad object is passed, PyCapsule_GetPointer() raises an ex‐ception, but Cython already knows to look for it and propagate it out of the distance() function if it occurs.A downside to the handling of Point structures is that they will be completely opaquein this implementation. You won’t be able to peek inside or access any of their attributes.There is an alternative approach to wrapping, which is to define an extension type, asshown in this code:
# sample.pyx
cimport csamplefrom libc.stdlib cimport malloc, free...
cdef class Point:
cdef csample.Point [*](#)_c_pointdef __cinit__(self, double x, double y):
> self._c_point = <csample.Point [*](#)> malloc(sizeof(csample.Point))self._c_point.x = xself._c_point.y = y
def __dealloc__(self):free(self._c_point)property x:def __get__(self):return self._c_point.xdef __set__(self, value):self._c_point.x = valueproperty y:def __get__(self):return self._c_point.ydef __set__(self, value):self._c_point.y = valuedef distance(Point p1, Point p2):return csample.distance(p1._c_point, p2._c_point)
Here, the cdef class Point is declaring Point as an extension type. The class variablecdef csample.Point [*](#)_c_point is declaring an instance variable that holds a pointerto an underlying Point structure in C. The __cinit__() and __dealloc__() methodscreate and destroy the underlying C structure using malloc() and free() calls. Theproperty x and property y declarations give code that gets and sets the underlyingstructure attributes. The wrapper for distance() has also been suitably modified toaccept instances of the Point extension type as arguments, but pass the underlyingpointer to the C function.Making this change, you will find that the code for manipulating Point objects is morenatural:
>>> import sample
>>> p1 = sample.Point(2,3)
>>> p2 = sample.Point(4,5)
>>> p1
<sample.Point object at 0x100447288>
>>> p2
<sample.Point object at 0x1004472a0>
>>> p1.x
2.0
>>> p1.y
3.0
>>> sample.distance(p1,p2)
2.8284271247461903
>>>
This recipe has illustrated many of Cython’s core features that you might be able toextrapolate to more complicated kinds of wrapping. However, you will definitely wantto read more of the official documentation to do more.The next few recipes also illustrate a few additional Cython features.
- Copyright
- 前言
- 第一章:數據結構和算法
- 1.1 解壓序列賦值給多個變量
- 1.2 解壓可迭代對象賦值給多個變量
- 1.3 保留最后N個元素
- 1.4 查找最大或最小的N個元素
- 1.5 實現一個優先級隊列
- 1.6 字典中的鍵映射多個值
- 1.7 字典排序
- 1.8 字典的運算
- 1.9 查找兩字典的相同點
- 1.10 刪除序列相同元素并保持順序
- 1.11 命名切片
- 1.12 序列中出現次數最多的元素
- 1.13 通過某個關鍵字排序一個字典列表
- 1.14 排序不支持原生比較的對象
- 1.15 通過某個字段將記錄分組
- 1.16 過濾序列元素
- 1.17 從字典中提取子集
- 1.18 映射名稱到序列元素
- 1.19 轉換并同時計算數據
- 1.20 合并多個字典或映射
- 第二章:字符串和文本
- 2.1 使用多個界定符分割字符串
- 2.2 字符串開頭或結尾匹配
- 2.3 用Shell通配符匹配字符串
- 2.4 字符串匹配和搜索
- 2.5 字符串搜索和替換
- 2.6 字符串忽略大小寫的搜索替換
- 2.7 最短匹配模式
- 2.8 多行匹配模式
- 2.9 將Unicode文本標準化
- 2.10 在正則式中使用Unicode
- 2.11 刪除字符串中不需要的字符
- 2.12 審查清理文本字符串
- 2.13 字符串對齊
- 2.14 合并拼接字符串
- 2.15 字符串中插入變量
- 2.16 以指定列寬格式化字符串
- 2.17 在字符串中處理html和xml
- 2.18 字符串令牌解析
- 2.19 實現一個簡單的遞歸下降分析器
- 2.20 字節字符串上的字符串操作
- 第三章:數字日期和時間
- 3.1 數字的四舍五入
- 3.2 執行精確的浮點數運算
- 3.3 數字的格式化輸出
- 3.4 二八十六進制整數
- 3.5 字節到大整數的打包與解包
- 3.6 復數的數學運算
- 3.7 無窮大與NaN
- 3.8 分數運算
- 3.9 大型數組運算
- 3.10 矩陣與線性代數運算
- 3.11 隨機選擇
- 3.12 基本的日期與時間轉換
- 3.13 計算最后一個周五的日期
- 3.14 計算當前月份的日期范圍
- 3.15 字符串轉換為日期
- 3.16 結合時區的日期操作
- 第四章:迭代器與生成器
- 4.1 手動遍歷迭代器
- 4.2 代理迭代
- 4.3 使用生成器創建新的迭代模式
- 4.4 實現迭代器協議
- 4.5 反向迭代
- 4.6 帶有外部狀態的生成器函數
- 4.7 迭代器切片
- 4.8 跳過可迭代對象的開始部分
- 4.9 排列組合的迭代
- 4.10 序列上索引值迭代
- 4.11 同時迭代多個序列
- 4.12 不同集合上元素的迭代
- 4.13 創建數據處理管道
- 4.14 展開嵌套的序列
- 4.15 順序迭代合并后的排序迭代對象
- 4.16 迭代器代替while無限循環
- 第五章:文件與IO
- 5.1 讀寫文本數據
- 5.2 打印輸出至文件中
- 5.3 使用其他分隔符或行終止符打印
- 5.4 讀寫字節數據
- 5.5 文件不存在才能寫入
- 5.6 字符串的I/O操作
- 5.7 讀寫壓縮文件
- 5.8 固定大小記錄的文件迭代
- 5.9 讀取二進制數據到可變緩沖區中
- 5.10 內存映射的二進制文件
- 5.11 文件路徑名的操作
- 5.12 測試文件是否存在
- 5.13 獲取文件夾中的文件列表
- 5.14 忽略文件名編碼
- 5.15 打印不合法的文件名
- 5.16 增加或改變已打開文件的編碼
- 5.17 將字節寫入文本文件
- 5.18 將文件描述符包裝成文件對象
- 5.19 創建臨時文件和文件夾
- 5.20 與串行端口的數據通信
- 5.21 序列化Python對象
- 第六章:數據編碼和處理
- 6.1 讀寫CSV數據
- 6.2 讀寫JSON數據
- 6.3 解析簡單的XML數據
- 6.4 增量式解析大型XML文件
- 6.5 將字典轉換為XML
- 6.6 解析和修改XML
- 6.7 利用命名空間解析XML文檔
- 6.8 與關系型數據庫的交互
- 6.9 編碼和解碼十六進制數
- 6.10 編碼解碼Base64數據
- 6.11 讀寫二進制數組數據
- 6.12 讀取嵌套和可變長二進制數據
- 6.13 數據的累加與統計操作
- 第七章:函數
- 7.1 可接受任意數量參數的函數
- 7.2 只接受關鍵字參數的函數
- 7.3 給函數參數增加元信息
- 7.4 返回多個值的函數
- 7.5 定義有默認參數的函數
- 7.6 定義匿名或內聯函數
- 7.7 匿名函數捕獲變量值
- 7.8 減少可調用對象的參數個數
- 7.9 將單方法的類轉換為函數
- 7.10 帶額外狀態信息的回調函數
- 7.11 內聯回調函數
- 7.12 訪問閉包中定義的變量
- 第八章:類與對象
- 8.1 改變對象的字符串顯示
- 8.2 自定義字符串的格式化
- 8.3 讓對象支持上下文管理協議
- 8.4 創建大量對象時節省內存方法
- 8.5 在類中封裝屬性名
- 8.6 創建可管理的屬性
- 8.7 調用父類方法
- 8.8 子類中擴展property
- 8.9 創建新的類或實例屬性
- 8.10 使用延遲計算屬性
- 8.11 簡化數據結構的初始化
- 8.12 定義接口或者抽象基類
- 8.13 實現數據模型的類型約束
- 8.14 實現自定義容器
- 8.15 屬性的代理訪問
- 8.16 在類中定義多個構造器
- 8.17 創建不調用init方法的實例
- 8.18 利用Mixins擴展類功能
- 8.19 實現狀態對象或者狀態機
- 8.20 通過字符串調用對象方法
- 8.21 實現訪問者模式
- 8.22 不用遞歸實現訪問者模式
- 8.23 循環引用數據結構的內存管理
- 8.24 讓類支持比較操作
- 8.25 創建緩存實例
- 第九章:元編程
- 9.1 在函數上添加包裝器
- 9.2 創建裝飾器時保留函數元信息
- 9.3 解除一個裝飾器
- 9.4 定義一個帶參數的裝飾器
- 9.5 可自定義屬性的裝飾器
- 9.6 帶可選參數的裝飾器
- 9.7 利用裝飾器強制函數上的類型檢查
- 9.8 將裝飾器定義為類的一部分
- 9.9 將裝飾器定義為類
- 9.10 為類和靜態方法提供裝飾器
- 9.11 裝飾器為被包裝函數增加參數
- 9.12 使用裝飾器擴充類的功能
- 9.13 使用元類控制實例的創建
- 9.14 捕獲類的屬性定義順序
- 9.15 定義有可選參數的元類
- 9.16 *args和**kwargs的強制參數簽名
- 9.17 在類上強制使用編程規約
- 9.18 以編程方式定義類
- 9.19 在定義的時候初始化類的成員
- 9.20 利用函數注解實現方法重載
- 9.21 避免重復的屬性方法
- 9.22 定義上下文管理器的簡單方法
- 9.23 在局部變量域中執行代碼
- 9.24 解析與分析Python源碼
- 9.25 拆解Python字節碼
- 第十章:模塊與包
- 10.1 構建一個模塊的層級包
- 10.2 控制模塊被全部導入的內容
- 10.3 使用相對路徑名導入包中子模塊
- 10.4 將模塊分割成多個文件
- 10.5 利用命名空間導入目錄分散的代碼
- 10.6 重新加載模塊
- 10.7 運行目錄或壓縮文件
- 10.8 讀取位于包中的數據文件
- 10.9 將文件夾加入到sys.path
- 10.10 通過字符串名導入模塊
- 10.11 通過導入鉤子遠程加載模塊
- 10.12 導入模塊的同時修改模塊
- 10.13 安裝私有的包
- 10.14 創建新的Python環境
- 10.15 分發包
- 第十一章:網絡與Web編程
- 11.1 作為客戶端與HTTP服務交互
- 11.2 創建TCP服務器
- 11.3 創建UDP服務器
- 11.4 通過CIDR地址生成對應的IP地址集
- 11.5 生成一個簡單的REST接口
- 11.6 通過XML-RPC實現簡單的遠程調用
- 11.7 在不同的Python解釋器之間交互
- 11.8 實現遠程方法調用
- 11.9 簡單的客戶端認證
- 11.10 在網絡服務中加入SSL
- 11.11 進程間傳遞Socket文件描述符
- 11.12 理解事件驅動的IO
- 11.13 發送與接收大型數組
- 第十二章:并發編程
- 12.1 啟動與停止線程
- 12.2 判斷線程是否已經啟動
- 12.3 線程間的通信
- 12.4 給關鍵部分加鎖
- 12.5 防止死鎖的加鎖機制
- 12.6 保存線程的狀態信息
- 12.7 創建一個線程池
- 12.8 簡單的并行編程
- 12.9 Python的全局鎖問題
- 12.10 定義一個Actor任務
- 12.11 實現消息發布/訂閱模型
- 12.12 使用生成器代替線程
- 12.13 多個線程隊列輪詢
- 12.14 在Unix系統上面啟動守護進程
- 第十三章:腳本編程與系統管理
- 13.1 通過重定向/管道/文件接受輸入
- 13.2 終止程序并給出錯誤信息
- 13.3 解析命令行選項
- 13.4 運行時彈出密碼輸入提示
- 13.5 獲取終端的大小
- 13.6 執行外部命令并獲取它的輸出
- 13.7 復制或者移動文件和目錄
- 13.8 創建和解壓壓縮文件
- 13.9 通過文件名查找文件
- 13.10 讀取配置文件
- 13.11 給簡單腳本增加日志功能
- 13.12 給內庫增加日志功能
- 13.13 記錄程序執行的時間
- 13.14 限制內存和CPU的使用量
- 13.15 啟動一個WEB瀏覽器
- 第十四章:測試調試和異常
- 14.1 測試輸出到標準輸出上
- 14.2 在單元測試中給對象打補丁
- 14.3 在單元測試中測試異常情況
- 14.4 將測試輸出用日志記錄到文件中
- 14.5 忽略或者期望測試失敗
- 14.6 處理多個異常
- 14.7 捕獲所有異常
- 14.8 創建自定義異常
- 14.9 捕獲異常后拋出另外的異常
- 14.10 重新拋出最后的異常
- 14.11 輸出警告信息
- 14.12 調試基本的程序崩潰錯誤
- 14.13 給你的程序做基準測試
- 14.14 讓你的程序跑的更快
- 第十五章:C語言擴展
- 15.1 使用ctypes訪問C代碼
- 15.2 簡單的C擴展模塊
- 15.3 一個操作數組的擴展函數
- 15.4 在C擴展模塊中操作隱形指針
- 15.5 從擴張模塊中定義和導出C的API
- 15.6 從C語言中調用Python代碼
- 15.7 從C擴展中釋放全局鎖
- 15.8 C和Python中的線程混用
- 15.9 用WSIG包裝C代碼
- 15.10 用Cython包裝C代碼
- 15.11 用Cython寫高性能的數組操作
- 15.12 將函數指針轉換為可調用對象
- 15.13 傳遞NULL結尾的字符串給C函數庫
- 15.14 傳遞Unicode字符串給C函數庫
- 15.15 C字符串轉換為Python字符串
- 15.16 不確定編碼格式的C字符串
- 15.17 傳遞文件名給C擴展
- 15.18 傳遞已打開的文件給C擴展
- 15.19 從C語言中讀取類文件對象
- 15.20 處理C語言中的可迭代對象
- 15.21 診斷分析代碼錯誤
- 附錄A
- 關于譯者
- Roadmap