python 处理 nginx 访问日志 demo

2018-08-01

108

0

技术:python2.7

运行环境:python2.7 + windows10

demo功能:提供一个python处理nginx 访问日志的代码demo

源代码

# coding:utf8

import json
from timeit import timeit

LOG_FILE = 'access.log-20180720.log'


def main():
    res = []
    with open(LOG_FILE, 'r') as lines:
        for line in lines:
            line_arr = line.split(' "')
            ip_and_times = line_arr[0].split()
            url_and_result_code = line_arr[1].split()
            if (len(res) == 100):
                output(res)
                res = []

            res.append({
                'ip': ip_and_times[0],
                'time': ip_and_times[3].replace('[', ''),
                'url': url_and_result_code[1],
                'code': url_and_result_code[3],
                'from': line_arr[3]
                    .replace('"', '')
                    .replace('\\', '')
            })


def output(list):
    print json.dumps(list, indent=4)
    # print len(list)


main()
# timeit(函数名_字符串,运行环境_字符串,number=运行次数)
# t = timeit('main()', '', number=1)
# print(t)
# nginx access log: 1.34GB
# 100000000 loops, best of 3: 0.00896 usec per loop
# 3.80853293159

结果

 

欢迎添加微信,互相学习↑↑↑ -_-

发表评论

全部评论:0条

白老虎

programming is not only to solve problems, ways to think