gil | 易学教程

python与GIL

阅读更多关于 python与GIL

一、GIL是什么，它为什么会存在　　python被人诟病最多的大概就是它的GIL全局锁了，但我今天要为它正名，这与python本身没有关系，而是与它的解释器有关系。　　我们知道，python代码编写好，最终运行的是一个.py的文件，运行代码的过程，其实就是让解释器把我们编写的代码解释给机器能识别的语言的过程。可以解释python代码的解释器有很多，如： cpython(官方解释器) ipython(基于cpython的一种解释器，交互更强，其他与cpython一样) pypy(采用JIT(Just In Time)也就是即时编译编译器,对Python代码执行动态编译,目的是加快执行速度) jython(Java平台上的解释器) ironpython(与jython类似，只不过ironPython是运行在微软.Net平台上的Python解释器) 　　官方使用的解释器是cpython，也就是当我们从官网下载安装python的时候，就已经默认使用python解释器是cpython。而GIL就是cpython在解释python代码的时候给它加的一个锁，GIL全称是Global Interpreter Lock(全局解释器锁)，所以GIL与cpython有关，与python无关，你可以换一种没有GIL的解释器。　　那么GIL为什么会存在呢，官网的解释是：GIL是一个互斥锁

Compilation error when using the move method in Cython

阅读更多关于 Compilation error when using the move method in Cython

问题 My question is similar to the one asked here - Passing C++ vector to Numpy through Cython without copying and taking care of memory management automatically I am also getting a segmentation fault but before I can fix it I have some compilation errors to take of with the move method in Cython. This is the simplest example I have (just an extension of the Rectangle example provided in the net What I want to do My C++ code returns a deque of Points. It can return a vector of points as well. Any

Numba `nogil=True` + ThreadPoolExecutor results in smaller speed up than expected

阅读更多关于 Numba `nogil=True` + ThreadPoolExecutor results in smaller speed up than expected

问题 This is a follow-up to my previous question: I'm trying to use Numba and Dask to speed up a slow computation that is similar to calculating the kernel density estimate of a huge collection of points. My plan was to write the computationally expensive logic in a jit ed function and then split the work among the CPU cores using dask . I wanted to use the nogil feature of numba.jit function so that I could use the dask threading backend so as to avoid unnecessary memory copies of the input data

Running computation in background thread python/pygtk

阅读更多关于 Running computation in background thread python/pygtk

问题 Is there a way to run a python thread in the background without locking down the rest of python during time-consuming instructions? I'm trying to do time-consuming calculations in a background thread of a python (pygtk) application. I understand how threads work. The problem is that each time I run an expensive operation in any thread (example: PIL's image.load() for large images), it blocks all python threads until the operation is completed, even though it is in a separate thread. So, is

How to avoid gcc warning in Python C extension when using Py_BEGIN_ALLOW_THREADS

阅读更多关于 How to avoid gcc warning in Python C extension when using Py_BEGIN_ALLOW_THREADS

问题 The simplest way to manipulate the GIL in Python C extensions is to use the macros provided: my_awesome_C_function() { blah; Py_BEGIN_ALLOW_THREADS // do stuff that doesn't need the GIL if (should_i_call_back) { Py_BLOCK_THREADS // do stuff that needs the GIL Py_UNBLOCK_THREADS } Py_END_ALLOW_THREADS return blah blah; } This works great, letting me release the GIL for the bulk of my code, but re-grabbing it for small bits of code that need it. The problem is when I compile this with gcc, I

GIL全局解释器锁

阅读更多关于 GIL全局解释器锁

拓展什么是GIL全局解释器锁为什么要有GIL全局解释器锁？ GIL全局解释器锁的优缺点多线程和IO密集型的应用场景转载文章 TOC 拓展 python解释器: Cpython C语言写的 Jpython java语言写的什么是GIL全局解释器锁在同一个进程下开启的多线程，同一时刻只能有一个线程执行，因为Cpython的内存管理不是线程安全摘自官方文档解释： In CPython, the global interpreter lock, or GIL, is a mutex that prevents multiple native threads from executing Python bytecodes at once. This lock is necessary mainly because CPython’s memory management is not thread-safe. (However, since the GIL exists, other features have grown to depend on the guarantees that it enforces.) 结论：在Cpython解释器中，同一个进程下开启的多线程，同一时刻只能有一个线程执行，无法利用多核优势。为什么要有GIL全局解释器锁？

python 线程，GIL 和 ctypes

阅读更多关于 python 线程，GIL 和 ctypes

GIL 与 Python 线程的纠葛 GIL 是什么东西？它对我们的 python 程序会产生什么样的影响？我们先来看一个问题。运行下面这段 python 程序，CPU 占用率是多少？请勿在工作中模仿，危险:) def dead_loop(): while True: pass dead_loop() 答案是什么呢，占用 100％ CPU？那是单核！还得是没有超线程的古董 CPU。在我的双核 CPU 上，这个死循环只会吃掉我一个核的工作负荷，也就是只占用 50％ CPU。那如何能让它在双核机器上占用 100％的 CPU 呢？答案很容易想到，用两个线程就行了，线程不正是并发分享 CPU 运算资源的吗。可惜答案虽然对了，但做起来可没那么简单。下面的程序在主线程之外又起了一个死循环的线程 import threading def dead_loop(): while True: pass 新起一个死循环线程 t = threading.Thread(target=dead_loop) t.start() 主线程也进入死循环 dead_loop() t.join() 按道理它应该能做到占用两个核的 CPU 资源，可是实际运行情况却是没有什么改变，还是只占了 50％ CPU 不到。这又是为什么呢？难道 python 线程不是操作系统的原生线程？打开 system monitor 一探究竟

Where's the GIL in PyPy?

阅读更多关于 Where's the GIL in PyPy?

问题 Is the PyPy GIL part of the PyPy interpreter implementation in RPython, or is it something that translate.py automatically adds? i.e., if I were to write my own new language interpreter in RPython and ran it through translate.py, would it be subject to the GIL a priori, or would that be up to my interpreter code? 回答1: The GIL handling is inserted by module/thread/gil.py in your PyPy checkout. It's an optional translation feature and it's only added when thread module is enabled. That said,

Python requires a GIL. But Jython & IronPython don't. Why?

阅读更多关于 Python requires a GIL. But Jython & IronPython don't. Why?

问题 Why is it that you can run Jython and IronPython without the need for a GIL but Python (CPython) requires a GIL? 回答1: Parts of the Interpreter aren't threadsafe, though mostly because making them all threadsafe by massive lock usage would slow single-threaded extremely (source). This seems to be related to the CPython garbage collector using reference counting (the JVM and CLR don't, and therefore don't need to lock/release a reference count every time). But even if someone thought of an

400 threads in 20 processes outperform 400 threads in 4 processes while performing a CPU-bound task on 4 CPUs

阅读更多关于 400 threads in 20 processes outperform 400 threads in 4 processes while performing a CPU-bound task on 4 CPUs

This question is very similar to 400 threads in 20 processes outperform 400 threads in 4 processes while performing an I/O-bound task . The only difference is that the linked question is about an I/O-bound task whereas this question is about a CPU-bound task. Experimental Code Here is the experimental code that can launch a specified number of worker processes and then launch a specified number of worker threads within each process and perform the task of computing the n-th prime number. import math import multiprocessing import random import sys import time import threading def main():

订阅 gil