爲什麼[]比list()快?

本文翻譯自:Why is [] faster than list()?

I recently compared the processing speeds of [] and list() and was surprised to discover that [] runs more than three times faster than list() . 我最近比較了[]list()的處理速度,並驚訝地發現[]運行速度list() 三倍。 I ran the same test with {} and dict() and the results were practically identical: [] and {} both took around 0.128sec / million cycles, while list() and dict() took roughly 0.428sec / million cycles each. 我使用{}dict()進行了相同的測試,結果實際上是相同的: []{}都花費了約0.128sec /百萬個週期,而list()dict()花費了約0.428sec /百萬個週期。

Why is this? 爲什麼是這樣? Do [] and {} (and probably () and '' , too) immediately pass back a copies of some empty stock literal while their explicitly-named counterparts ( list() , dict() , tuple() , str() ) fully go about creating an object, whether or not they actually have elements? Do []{} (可能還有()'' )也會立即傳回一些空股票文字的副本,同時使用它們的顯式名稱副本( list()dict()tuple()str() )完全去創建一個對象,無論它們實際上是否包含元素?

I have no idea how these two methods differ but I'd love to find out. 我不知道這兩種方法有何不同,但我很想找出答案。 I couldn't find an answer in the docs or on SO, and searching for empty brackets turned out to be more problematic than I'd expected. 我在文檔中或SO上都找不到答案,而尋找空括號卻比我預期的要麻煩得多。

I got my timing results by calling timeit.timeit("[]") and timeit.timeit("list()") , and timeit.timeit("{}") and timeit.timeit("dict()") , to compare lists and dictionaries, respectively. 我通過調用timeit.timeit("[]")timeit.timeit("list()")以及timeit.timeit("{}")timeit.timeit("dict()")獲得計時結果。分別比較列表和字典。 I'm running Python 2.7.9. 我正在運行Python 2.7.9。

I recently discovered " Why is if True slower than if 1? " that compares the performance of if True to if 1 and seems to touch on a similar literal-versus-global scenario; 我最近發現“ 爲什麼True慢於if 1? ”將if Trueif 1的性能進行了比較,似乎觸及了類似的文字對全局的情況。 perhaps it's worth considering as well. 也許也值得考慮。


#1樓

參考:https://stackoom.com/question/22mYq/爲什麼-比list-快


#2樓

Because list is a function to convert say a string to a list object, while [] is used to create a list off the bat. 由於list是一個功能轉化說一個字符串列表對象,而[]用於創建一個列表蝙蝠。 Try this (might make more sense to you): 嘗試一下(可能對您更有意義):

x = "wham bam"
a = list(x)
>>> a
["w", "h", "a", "m", ...]

While

y = ["wham bam"]
>>> y
["wham bam"]

Gives you a actual list containing whatever you put in it. 爲您提供包含您所輸入內容的實際列表。


#3樓

Because [] and {} are literal syntax . 因爲[]{}文字語法 Python can create bytecode just to create the list or dictionary objects: Python可以創建字節碼僅用於創建列表或字典對象:

>>> import dis
>>> dis.dis(compile('[]', '', 'eval'))
  1           0 BUILD_LIST               0
              3 RETURN_VALUE        
>>> dis.dis(compile('{}', '', 'eval'))
  1           0 BUILD_MAP                0
              3 RETURN_VALUE        

list() and dict() are separate objects. list()dict()是單獨的對象。 Their names need to be resolved, the stack has to be involved to push the arguments, the frame has to be stored to retrieve later, and a call has to be made. 它們的名稱需要解析,必須包含堆棧以推入參數,必須存儲框架以供以後檢索,並且必須進行調用。 That all takes more time. 這都需要更多時間。

For the empty case, that means you have at the very least a LOAD_NAME (which has to search through the global namespace as well as the __builtin__ module ) followed by a CALL_FUNCTION , which has to preserve the current frame: 對於空的情況,這意味着您至少有一個LOAD_NAME (必須搜索全局名稱空間以及__builtin__模塊 ),然後是CALL_FUNCTION ,後者必須保留當前幀:

>>> dis.dis(compile('list()', '', 'eval'))
  1           0 LOAD_NAME                0 (list)
              3 CALL_FUNCTION            0
              6 RETURN_VALUE        
>>> dis.dis(compile('dict()', '', 'eval'))
  1           0 LOAD_NAME                0 (dict)
              3 CALL_FUNCTION            0
              6 RETURN_VALUE        

You can time the name lookup separately with timeit : 您可以使用timeit分別對名稱查找進行計時:

>>> import timeit
>>> timeit.timeit('list', number=10**7)
0.30749011039733887
>>> timeit.timeit('dict', number=10**7)
0.4215109348297119

The time discrepancy there is probably a dictionary hash collision. 時間差異可能是字典哈希衝突。 Subtract those times from the times for calling those objects, and compare the result against the times for using literals: 從調用這些對象的時間中減去這些時間,然後將結果與使用文字的時間進行比較:

>>> timeit.timeit('[]', number=10**7)
0.30478692054748535
>>> timeit.timeit('{}', number=10**7)
0.31482696533203125
>>> timeit.timeit('list()', number=10**7)
0.9991960525512695
>>> timeit.timeit('dict()', number=10**7)
1.0200958251953125

So having to call the object takes an additional 1.00 - 0.31 - 0.30 == 0.39 seconds per 10 million calls. 因此,每1千萬次調用必須調用對象額外花費1.00 - 0.31 - 0.30 == 0.39秒。

You can avoid the global lookup cost by aliasing the global names as locals (using a timeit setup, everything you bind to a name is a local): 您可以通過將全局名稱別名爲本地名稱來避免全局查找成本(使用timeit設置,您綁定到名稱的所有內容都是本地名稱):

>>> timeit.timeit('_list', '_list = list', number=10**7)
0.1866450309753418
>>> timeit.timeit('_dict', '_dict = dict', number=10**7)
0.19016098976135254
>>> timeit.timeit('_list()', '_list = list', number=10**7)
0.841480016708374
>>> timeit.timeit('_dict()', '_dict = dict', number=10**7)
0.7233691215515137

but you never can overcome that CALL_FUNCTION cost. 但您永遠無法克服CALL_FUNCTION費用。


#4樓

list() requires a global lookup and a function call but [] compiles to a single instruction. list()需要全局查找和函數調用,但是[]編譯爲一條指令。 See: 看到:

Python 2.7.3
>>> import dis
>>> print dis.dis(lambda: list())
  1           0 LOAD_GLOBAL              0 (list)
              3 CALL_FUNCTION            0
              6 RETURN_VALUE        
None
>>> print dis.dis(lambda: [])
  1           0 BUILD_LIST               0
              3 RETURN_VALUE        
None

#5樓

The answers here are great, to the point and fully cover this question. 至此,答案非常好,並完全涵蓋了這個問題。 I'll drop a further step down from byte-code for those interested. 對於那些感興趣的人,我將進一步從字節碼中刪除。 I'm using the most recent repo of CPython; 我正在使用CPython的最新倉庫; older versions behave similar in this regard but slight changes might be in place. 在這方面,舊版本的行爲類似,但可能會稍作更改。

Here's a break down of the execution for each of these, BUILD_LIST for [] and CALL_FUNCTION for list() . 這是每個命令執行的BUILD_LIST[] CALL_FUNCTION [] CALL_FUNCTIONlist() CALL_FUNCTION


The BUILD_LIST instruction: BUILD_LIST指令:

You should just view the horror: 您應該只查看恐怖:

PyObject *list =  PyList_New(oparg);
if (list == NULL)
    goto error;
while (--oparg >= 0) {
    PyObject *item = POP();
    PyList_SET_ITEM(list, oparg, item);
}
PUSH(list);
DISPATCH();

Terribly convoluted, I know. 我知道那令人費解。 This is how simple it is: 這是多麼簡單:

  • Create a new list with PyList_New (this mainly allocates the memory for a new list object), oparg signalling the number of arguments on the stack. 使用PyList_New創建一個新列表(這主要爲新的列表對象分配內存), oparg指示堆棧上的參數數量。 Straight to the point. 開門見山。
  • Check that nothing went wrong with if (list==NULL) . 檢查if (list==NULL)
  • Add any arguments (in our case this isn't executed) located on the stack with PyList_SET_ITEM (a macro). 使用PyList_SET_ITEM (一個宏)添加位於堆棧上的所有參數(在我們的示例中不執行)。

No wonder it is fast! 難怪它很快! It's custom-made for creating new lists, nothing else :-) 它是爲創建新列表而定製的,僅此而已:-)

The CALL_FUNCTION instruction: CALL_FUNCTION指令:

Here's the first thing you see when you peek at the code handling CALL_FUNCTION : 窺視代碼處理CALL_FUNCTION時,這是您看到的第一件事:

PyObject **sp, *res;
sp = stack_pointer;
res = call_function(&sp, oparg, NULL);
stack_pointer = sp;
PUSH(res);
if (res == NULL) {
    goto error;
}
DISPATCH();

Looks pretty harmless, right? 看起來很無害吧? Well, no, unfortunately not, call_function is not a straightforward guy that will call the function immediately, it can't. 好吧,不,不幸的是, call_function不是一個直截了當的傢伙,它不會立即調用該函數。 Instead, it grabs the object from the stack, grabs all arguments of the stack and then switches based on the type of the object; 相反,它從堆棧中獲取對象,獲取堆棧中的所有參數,然後根據對象的類型進行切換。 is it a: 它是:

We're calling the list type, the argument passed in to call_function is PyList_Type . 我們正在調用list類型,傳遞給call_function的參數是PyList_Type CPython now has to call a generic function to handle any callable objects named _PyObject_FastCallKeywords , yay more function calls. CPython現在必須調用一個泛型函數來處理名爲_PyObject_FastCallKeywords所有可調用對象,還需要更多函數調用。

This function again makes some checks for certain function types (which I cannot understand why) and then, after creating a dict for kwargs if required , goes on to call _PyObject_FastCallDict . 該函數再次檢查某些函數類型(我不明白爲什麼),然後在爲kwargs創建字典之後, 如果需要 ,繼續調用_PyObject_FastCallDict

_PyObject_FastCallDict finally gets us somewhere! _PyObject_FastCallDict終於使我們到了某個地方! After performing even more checks it grabs the tp_call slot from the type of the type we've passed in, that is, it grabs type.tp_call . 在執行更多檢查之後,我們傳入的type type搶佔了tp_call插槽 ,即,它type.tp_call It then proceeds to create a tuple out of of the arguments passed in with _PyStack_AsTuple and, finally, a call can finally be made ! 然後, _PyStack_AsTuple根據_PyStack_AsTuple傳入的參數來創建元組,最後可以最終進行調用

tp_call , which matches type.__call__ takes over and finally creates the list object. tp_call ,它與type.__call__匹配type.__call__接管並最終創建列表對象。 It calls the lists __new__ which corresponds to PyType_GenericNew and allocates memory for it with PyType_GenericAlloc : This is actually the part where it catches up with PyList_New , finally . 它調用對應於PyType_GenericNew的列表__new__ ,並使用PyType_GenericAlloc分配內存: 實際上,這實際上是它最終趕上PyList_New的部分 All the previous are necessary to handle objects in a generic fashion. 所有以前的內容對於以通用方式處理對象都是必需的。

In the end, type_call calls list.__init__ and initializes the list with any available arguments, then we go on a returning back the way we came. 最後, type_call調用list.__init__並使用任何可用參數初始化該列表,然後繼續返回原來的方式。 :-) :-)

Finally, remmeber the LOAD_NAME , that's another guy that contributes here. 最後,記住LOAD_NAME ,這是另一個在這裏做出貢獻的傢伙。


It's easy to see that, when dealing with our input, Python generally has to jump through hoops in order to actually find out the appropriate C function to do the job. 很容易看到,在處理我們的輸入時,Python通常必須跳過圈,才能真正找到合適的C函數來完成這項工作。 It doesn't have the curtesy of immediately calling it because it's dynamic, someone might mask list ( and boy do many people do ) and another path must be taken. 它不具有立即調用它的功能,因爲它是動態的,有人可能會屏蔽list男孩會做很多人做的事情 ),因此必須採取另一條路徑。

This is where list() loses much: The exploring Python needs to do to find out what the heck it should do. 這是list()損失慘重的地方:不斷探索的Python需要做的事情才能弄清楚它應該做什麼。

Literal syntax, on the other hand, means exactly one thing; 另一方面,字面語法恰好意味着一回事。 it cannot be changed and always behaves in a pre-determined way. 它無法更改,並且始終以預定的方式運行。

Footnote: All function names are subject to change from one release to the other. 腳註:所有功能名稱均可能從一個版本更改爲另一個版本。 The point still stands and most likely will stand in any future versions, it's the dynamic look-up that slows things down. 關鍵點仍然存在,並且很可能在將來的任何版本中都存在,這是動態查找使事情變慢的原因。


#6樓

Why is [] faster than list() ? 爲什麼[]list()快?

The biggest reason is that Python treats list() just like a user-defined function, which means you can intercept it by aliasing something else to list and do something different (like use your own subclassed list or perhaps a deque). 最大的原因是Python像對待用戶定義的函數一樣對待list() ,這意味着您可以通過別名其他對象來list並執行其他操作(例如使用您自己的子類列表或雙端隊列)來攔截它。

It immediately creates a new instance of a builtin list with [] . 它立即使用[]創建內置列表的新實例。

My explanation seeks to give you the intuition for this. 我的解釋旨在爲您提供直覺。

Explanation 說明

[] is commonly known as literal syntax. []通常稱爲文字語法。

In the grammar, this is referred to as a "list display". 在語法中,這稱爲“列表顯示”。 From the docs : 從文檔

A list display is a possibly empty series of expressions enclosed in square brackets: 列表顯示是括在方括號中的一系列可能爲空的表達式:

 list_display ::= "[" [starred_list | comprehension] "]" 

A list display yields a new list object, the contents being specified by either a list of expressions or a comprehension. 列表顯示將產生一個新的列表對象,其內容由表達式列表或理解列表指定。 When a comma-separated list of expressions is supplied, its elements are evaluated from left to right and placed into the list object in that order. 提供逗號分隔的表達式列表時,將按從左到右的順序評估其元素,並將其按順序放入列表對象中。 When a comprehension is supplied, the list is constructed from the elements resulting from the comprehension. 提供理解後,將根據理解產生的元素來構建列表。

In short, this means that a builtin object of type list is created. 簡而言之,這意味着將創建類型爲list的內置對象。

There is no circumventing this - which means Python can do it as quickly as it may. 不能迴避這一點-這意味着Python可以儘快完成它。

On the other hand, list() can be intercepted from creating a builtin list using the builtin list constructor. 在另一方面, list()可以從創建一個內置的攔截list使用內置列表構造。

For example, say we want our lists to be created noisily: 例如,假設我們希望創建噪音較大的列表:

class List(list):
    def __init__(self, iterable=None):
        if iterable is None:
            super().__init__()
        else:
            super().__init__(iterable)
        print('List initialized.')

We could then intercept the name list on the module level global scope, and then when we create a list , we actually create our subtyped list: 然後,我們可以在模塊級別的全局範圍內攔截名稱list ,然後在創建list ,實際上創建了子類型列表:

>>> list = List
>>> a_list = list()
List initialized.
>>> type(a_list)
<class '__main__.List'>

Similarly we could remove it from the global namespace 同樣,我們可以將其從全局名稱空間中刪除

del list

and put it in the builtin namespace: 並將其放在內置名稱空間中:

import builtins
builtins.list = List

And now: 現在:

>>> list_0 = list()
List initialized.
>>> type(list_0)
<class '__main__.List'>

And note that the list display creates a list unconditionally: 並注意列表顯示無條件創建列表:

>>> list_1 = []
>>> type(list_1)
<class 'list'>

We probably only do this temporarily, so lets undo our changes - first remove the new List object from the builtins: 我們可能只是暫時執行此操作,所以請撤消更改-首先從內置文件中刪除新的List對象:

>>> del builtins.list
>>> builtins.list
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'builtins' has no attribute 'list'
>>> list()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
NameError: name 'list' is not defined

Oh, no, we lost track of the original. 哦,不,我們失去了原來的蹤跡。

Not to worry, we can still get list - it's the type of a list literal: 不用擔心,我們仍然可以獲得list -這是列表文字的類型:

>>> builtins.list = type([])
>>> list()
[]

So... 所以...

Why is [] faster than list() ? 爲什麼[]list()快?

As we've seen - we can overwrite list - but we can't intercept the creation of the literal type. 如我們所見-我們可以覆蓋list -但我們無法攔截文字類型的創建。 When we use list we have to do the lookups to see if anything is there. 當使用list我們必須進行查找以查看是否存在任何內容。

Then we have to call whatever callable we have looked up. 然後,我們必須調用已查找的任何可調用對象。 From the grammar: 從語法上:

A call calls a callable object (eg, a function) with a possibly empty series of arguments: 調用使用一系列可能爲空的參數來調用可調用對象(例如,函數):

 call ::= primary "(" [argument_list [","] | comprehension] ")" 

We can see that it does the same thing for any name, not just list: 我們可以看到它對任何名稱都具有相同的作用,而不僅僅是列表:

>>> import dis
>>> dis.dis('list()')
  1           0 LOAD_NAME                0 (list)
              2 CALL_FUNCTION            0
              4 RETURN_VALUE
>>> dis.dis('doesnotexist()')
  1           0 LOAD_NAME                0 (doesnotexist)
              2 CALL_FUNCTION            0
              4 RETURN_VALUE

For [] there is no function call at the Python bytecode level: 對於[] ,在Python字節碼級別沒有函數調用:

>>> dis.dis('[]')
  1           0 BUILD_LIST               0
              2 RETURN_VALUE

It simply goes straight to building the list without any lookups or calls at the bytecode level. 它只是直接建立列表而無需在字節碼級別進行任何查找或調用。

Conclusion 結論

We have demonstrated that list can be intercepted with user code using the scoping rules, and that list() looks for a callable and then calls it. 我們已經證明,可以使用範圍規則使用用戶代碼攔截list ,並且list()查找可調用對象,然後調用它。

Whereas [] is a list display, or a literal, and thus avoids the name lookup and function call. []是列表顯示或文字,因此避免了名稱查找和函數調用。

發表評論
所有評論
還沒有人評論,想成為第一個評論的人麼? 請在上方評論欄輸入並且點擊發布.
相關文章