解决 Mavericks 系统上 Python 库 - mno-fused-madd 错误

最近升级了一下 simplejson,发现 simplejson 的 speedup 模块报了一个错误:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
Installing collected packages: simplejson
Found existing installation: simplejson 3.3.1
Uninstalling simplejson:
Successfully uninstalled simplejson
Running setup.py install for simplejson
building 'simplejson._speedups' extension
cc -fno-strict-aliasing -fno-common -dynamic -arch x86_64 -arch i386 -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o
clang: error: unknown argument: '-mno-fused-madd' [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
***************************************************************************
WARNING: The C extension could not be compiled, speedups are not enabled.
Failure information, if any, is above.
I'm retrying the build without the C extension now.
***************************************************************************

***************************************************************************
WARNING: The C extension could not be compiled, speedups are not enabled.
Plain-Python installation succeeded.
***************************************************************************
Successfully installed simplejson
Cleaning up...

注意看 error 这里,有一个

argument: '-mno-fused-madd'``` 的错误,应该是 Xcode 升级最新的版本之后,clang 替换带来的问题。
1
2

解决方法也很简单,在编译之前 export 两个参数即可:

$ export CFLAGS=-Qunused-arguments
$ export CPPFLAGS=-Qunused-arguments
$ pip install simplejson
Downloading/unpacking simplejson
Downloading simplejson-3.4.0.tar.gz (68kB): 68kB downloaded
Running setup.py egg_info for package simplejson

Installing collected packages: simplejson
Running setup.py install for simplejson
building ‘simplejson._speedups’ extension
cc -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -Qunused-arguments -Qunused-arguments -arch x86_64 -arch i386 -pipe -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o
cc -bundle -undefined dynamic_lookup -arch x86_64 -arch i386 -Wl,-F. -Qunused-arguments -Qunused-arguments build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o -o build/lib.macosx-10.9-intel-2.7/simplejson/_speedups.so

Successfully installed simplejson
Cleaning up…

1
2

后来查了一下具体的原因,在 [Xcode5.1 的 release note](https://developer.apple.com/library/mac/releasenotes/DeveloperTools/RN-Xcode/Introduction/Introduction.html) 中有提及:

The Apple LLVM compiler in Xcode 5.1 treats unrecognized command-line options as errors. This issue has been seen when building both Python native extensions and Ruby Gems, where some invalid compiler options are currently specified.

Projects using invalid compiler options will need to be changed to remove those options. To help ease that transition, the compiler will temporarily accept an option to downgrade the error to a warning:

-Wno-error=unused-command-line-argument-hard-error-in-future

1
2

官方推荐的方法是:

$ ARCHFLAGS=-Wno-error=unused-command-line-argument-hard-error-in-future pip install simplejson
Downloading/unpacking simplejson
Downloading simplejson-3.4.0.tar.gz (68kB): 68kB downloaded
Running setup.py egg_info for package simplejson

Installing collected packages: simplejson
Running setup.py install for simplejson
building ‘simplejson._speedups’ extension
cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -mno-fused-madd -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -Wno-error=unused-command-line-argument-hard-error-in-future -pipe -Wno-error=unused-command-line-argument-hard-error-in-future -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c simplejson/_speedups.c -o build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o
clang: warning: unknown argument: ‘-mno-fused-madd’ [-Wunused-command-line-argument-hard-error-in-future]
clang: note: this will be a hard error (cannot be downgraded to a warning) in the future
clang: warning: argument unused during compilation: ‘-mno-fused-madd’
cc -bundle -undefined dynamic_lookup -Wl,-F. -Wno-error=unused-command-line-argument-hard-error-in-future -Wno-error=unused-command-line-argument-hard-error-in-future build/temp.macosx-10.9-intel-2.7/simplejson/_speedups.o -o build/lib.macosx-10.9-intel-2.7/simplejson/_speedups.so

Successfully installed simplejson
Cleaning up…

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
最后来个轻量级的性能测试 (python 2.7.5):

```python
#!/usr/bin/env python
# -*- coding: utf-8 -*-

from timeit import Timer
import json
import simplejson


def by_json():
a = {"a": 1, "b": "2", "c": [1, 2, 3], "d": ["1", "2", "3"]}
b = json.dumps(a)
a = json.loads(b)


def by_simplejson():
a = {"a": 1, "b": "2", "c": [1, 2, 3], "d": ["1", "2", "3"]}
b = simplejson.dumps(a)
a = simplejson.loads(b)


if __name__ == '__main__':
t = Timer(stmt="by_json()", setup="from __main__ import by_json")
print 'by json: %s seconds' % t.timeit(number=3)
t = Timer(stmt="by_simplejson()", setup="from __main__ import by_simplejson")
print 'by simplejson: %s seconds' % t.timeit(number=3)

测试结果表明:恩,实际上不用 simplejson 也可以,不会有很大性能差异:

1
2
3
$ python json_speed.py
by json: 0.000191926956177 seconds
by simplejson: 0.000169992446899 seconds