Python有哪些黑魔法?
回复内容:
ctypes ,当年有个人靠这个省了好几个月的加班------------------------劳动节补充-----------------------------
回答 @于酥酥
1. ipython + ctypes: 调试/测试Linux API的交互式运行环境
ipython是最好的REPL!(我喜欢Python,至少有30%的好感来自ipython)
REPL的好处不言自明,在开发和调试时能大大的提高效率。尤其是需要对API进行快速验证时。
我最早是用gdb来做一些REPL的事情,但毕竟操作复杂,交互式功能有限。而用ctypes,就爽快多了,ctypes可以直接人so中提取出函数,在Python层面稍加包装,就能直接使用,不用编译/连接,保持运行状态,结果出来直接用Python分析……简直是画面太美
真实场景:
我们的运营环境有数十万host,host上去除了编译环境,某一天,我们对某个系统调用的返回产生怀疑。于是,按照通常的作法,在开发机上写一个示例程序,编译,拷贝到运营机,运行,反复执行这一个过程。
那么有了ctypes,直接在python/ipython的REPL里调试就好了。还不容易留下可疑的可执行程序。
2. ctypes作为胶水
ctypes增强了python作为胶水语言的能力,从进程调用/统一协议级别的脱水直接深入到二进制级别的脱水。这样看来,C++对C的兼容就显得没那么重要了
真实场景:
某个执行框架,插件以so的形式提供,so提供固定的函数入口。重构时打算去除语言耦合,改用进程调用的方式调用插件(类似于cgi server,这样可以减少对插件编写的限制,插件本身也更容易测试,防止so崩溃造成框架整体崩溃)。但是很多插件的作者已离职,于是只需要框架额外增加从so里调用函数出来即可,做到平滑升级。
3. ctypes与系统编程
ctypes作为一种轻量并且内置的c语言“代理”,使得python极大地增强了系统编程的能力。
从此,系统编程的代码也可以变得更加优雅。
真实场景:
sdn/vpc方案需要对内核协议栈做较多的调整,从管理的层面上,网络配置由中央控制并下发。因此,host上存在一个daemon,一方面要接受zookeeper的配置变更通知,另一方面要把配置解析后通过netlink与内核通信。
这个daemon大概几乎没有人会用python去做。但是我看到iotop里用到ctypes对netlink接口的封装,惊为天人,并且python更加适合对配置解析与处理。我斗胆用python实现了这个daemon,调试起来如丝般顺滑,然后就减少了好几个月的加班。
回答我成电师兄 @韦易笑 大神
没有有生产环境用过cffi,以前在自己电脑上简单用过,感觉不如ctypes简单粗暴。当然我没去用的主要原因还是不想在部署的时候附带太多东西。 在Quora上看到的, 不算黑科技吧, 但感觉挺有意思的. turtle是内置库


======================================================================
补充一个最近才看到的:
要对字典里面的键嵌套赋值, 对键不存在时候的解决方案:
<span class="kn">import</span> <span class="nn">collections</span> <span class="n">tree</span> <span class="o">=</span> <span class="k">lambda</span><span class="p">:</span> <span class="n">collections</span><span class="o">.</span><span class="n">defaultdict</span><span class="p">(</span><span class="n">tree</span><span class="p">)</span> <span class="n">some_dict</span> <span class="o">=</span> <span class="n">tree</span><span class="p">()</span> <span class="n">some_dict</span><span class="p">[</span><span class="s">"colors"</span><span class="p">][</span><span class="s">"favourite"</span><span class="p">]</span> <span class="o">=</span> <span class="s">"yellow"</span>
<span class="k">with</span> <span class="n">timeout</span><span class="p">(</span><span class="n">seconds</span><span class="o">=</span><span class="mi">10</span><span class="p">):</span> <span class="n">balabala</span><span class="p">()</span>
difflib,它是个official的module哦,用来比较串的相似度。
(difflib)[difflib – Compare sequences],
另外常用的functools和collections也都是Python吸引人的地方。
很多第三方的库窃以为不能算了。
另外doctest在进行单元测试的时候也是棒呆
25.2. doctest 说到python黑魔法,必然要提到python的第三方协程库gevent的底层实现——greenlet。
greenlet直接在内存层面,通过保存和替换Python进程的运行栈来实现不同协程的切换。
这个切换对于python解释器是透明的,如果python解释器对环境有感知的话,则每当协程切换的时候,它的感觉可能类似一个人前一秒还在在路上走路,下一秒突然自己又出现在了地铁上。
对于普通python用户而言,直接操作python的运行时栈,这就是在刀尖上跳舞有木有,这要求对内存的操作100%精确,任何错误都可能导致python进程崩溃!
那作者又是如何又是如何来保证正确性呢?除了要熟悉python、操作系统、编译器等等的底层机制,明确设计方案,还需要对不同的系统以及硬件环境做对应的适配工作。我们在使用python的时候,在不同的系统或者硬件下感觉都一样,那是因为python本身为我们屏蔽了底层细节,在做这种python底层hack的事情的时候,显然就没那么轻松了。
举个例子,由于CPU有很多种,例如i386、x86_64、arm等等,每种CPU的设计不尽相同,于是作者为每种CPU写了对应的汇编操作指令来完成栈的保存和替换,这些操作都是与操作系统和硬件高度绑定的。
虽然greenlet的实现这么bt,但就是有人做到了,加上gevent的封装,用起来比python自带协程好用太多。
我想任何对python比较熟悉的童鞋,在初次接触gevent的时候,都会好奇它是如何做到的,在进一步了解其底层greenlet实现机理之后,无不惊叹其鬼斧神工。
这种事情就是那种,别人不说,你可能永远不会想到的事情。 pip一下啥都有呀,比如微信接口 itchat
pip install itchat

- Flask 中的插件是怎么做的? 为何能用
from flask.ext.sqlalchemy import SQLAlchemy
Copy after login

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics











Python excels in gaming and GUI development. 1) Game development uses Pygame, providing drawing, audio and other functions, which are suitable for creating 2D games. 2) GUI development can choose Tkinter or PyQt. Tkinter is simple and easy to use, PyQt has rich functions and is suitable for professional development.

Python is easier to learn and use, while C is more powerful but complex. 1. Python syntax is concise and suitable for beginners. Dynamic typing and automatic memory management make it easy to use, but may cause runtime errors. 2.C provides low-level control and advanced features, suitable for high-performance applications, but has a high learning threshold and requires manual memory and type safety management.

To maximize the efficiency of learning Python in a limited time, you can use Python's datetime, time, and schedule modules. 1. The datetime module is used to record and plan learning time. 2. The time module helps to set study and rest time. 3. The schedule module automatically arranges weekly learning tasks.

Python is better than C in development efficiency, but C is higher in execution performance. 1. Python's concise syntax and rich libraries improve development efficiency. 2.C's compilation-type characteristics and hardware control improve execution performance. When making a choice, you need to weigh the development speed and execution efficiency based on project needs.

Pythonlistsarepartofthestandardlibrary,whilearraysarenot.Listsarebuilt-in,versatile,andusedforstoringcollections,whereasarraysareprovidedbythearraymoduleandlesscommonlyusedduetolimitedfunctionality.

Python excels in automation, scripting, and task management. 1) Automation: File backup is realized through standard libraries such as os and shutil. 2) Script writing: Use the psutil library to monitor system resources. 3) Task management: Use the schedule library to schedule tasks. Python's ease of use and rich library support makes it the preferred tool in these areas.

Is it enough to learn Python for two hours a day? It depends on your goals and learning methods. 1) Develop a clear learning plan, 2) Select appropriate learning resources and methods, 3) Practice and review and consolidate hands-on practice and review and consolidate, and you can gradually master the basic knowledge and advanced functions of Python during this period.

Python and C each have their own advantages, and the choice should be based on project requirements. 1) Python is suitable for rapid development and data processing due to its concise syntax and dynamic typing. 2)C is suitable for high performance and system programming due to its static typing and manual memory management.
