日志收集探讨
在项目上线后，往往需要监控一些日志，但是日志分散在各个服务器的不同角落，查看时需要找到某应用在那台服务器上，然后翻apache或nginx配置找日志路径，耗时耗力很麻烦。
所以想将这些日志收集到一个地方，做一套web界面的日志查看工具
需要的日志主要有
- apache日志
- apache虚拟主机access,error日志
- nginx日志
- nginx虚拟主机access,error日志
- php错误日志
- mysql日志
- php应用日志(如yii的/runtime/application.log)
基本实现实时查看即可（一分钟内延迟）
看了下Scribe和Flume，似乎太过重量级，
我的想法是做个简单的文本变化检测，将新增内容发送到统一redis队列，后端脚本处理入库。
不知道大家是如何解决这个问题的？探讨下吧！
回复内容：
在项目上线后，往往需要监控一些日志，但是日志分散在各个服务器的不同角落，查看时需要找到某应用在那台服务器上，然后翻apache或nginx配置找日志路径，耗时耗力很麻烦。
所以想将这些日志收集到一个地方，做一套web界面的日志查看工具
需要的日志主要有
- apache日志
- apache虚拟主机access,error日志
- nginx日志
- nginx虚拟主机access,error日志
- php错误日志
- mysql日志
- php应用日志(如yii的/runtime/application.log)
基本实现实时查看即可（一分钟内延迟）
看了下Scribe和Flume，似乎太过重量级，
我的想法是做个简单的文本变化检测，将新增内容发送到统一redis队列，后端脚本处理入库。
不知道大家是如何解决这个问题的？探讨下吧！
你的这个需求直接使用elasticsearch、logstash和kibana结合即可，部署配置都比较简单，实时性较高，又不用开发代码~
ELK全部搞定 妥妥的 https://www.elastic.co/
说一个常用方案：很多公司都是专门布了脚本，定时去抓，抓完了索引存到数据平台，数据平台提供查询，接口确实是略重。
来点前沿科技——systemd 的 journald～不用检测文本变化，它本身就是类似数据库那种二进制存储，直接用它给的 API 读取就可以了。当然目前 journald 不支持搜索，只能根据一些域来取数据，不过满足你的需求够了。
当然我猜测你们服务器上目前是不会用 systemd 的。journald 大概也不能不依赖 systemd 自己跑……
看一下logstash吧。 http://logstash.net/
负责任地说，成熟的日志处理和监控平台很多很多很多很多。。手机用户就不方便写了改天用电脑的时候给你列出来十几样
参考：在Linux上使用logwatch分析监控日志文件
看看Sentry？https://github.com/getsentry/sentry
go语言的话可以考虑一下heka
楼主可以关注下开源分布式搜索平台ELK(Elasticsearch+Logstash+Kibana)
http://finallygo.iteye.com/blog/2170588
建议你看看这个,非常轻量
成熟的商业方案Splunk不错。
fluentd + MongoDB
可以试试阿里云的日志服务，即开即用，无需运维。
1）提供客户端快速采集日志（https://yq.aliyun.com/articles/3228?spm=5176.team4.teamshow1.37.faH93Z）
2）日志收集到服务端后提供多种消费方式：数据通道（类Kafka）/数据投递（ODPS/OSS）/实时搜索
具体可以参考：https://help.aliyun.com/document_detail/sls/user-guide/overview.html
Hot AI Tools
Undresser.AI Undress
AI-powered app for creating realistic nude photos
AI Clothes Remover
Online AI tool for removing clothes from photos.
Undress AI Tool
Undress images for free
Clothoff.io
AI clothes remover
Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!
Hot Article
Hot Tools
Notepad++7.3.1
Easy-to-use and free code editor
SublimeText3 Chinese version
Chinese version, very easy to use
Zend Studio 13.0.1
Powerful PHP integrated development environment
Dreamweaver CS6
Visual web development tools
SublimeText3 Mac version
God-level code editing software (SublimeText3)
Hot Topics
1669
14
1428
52
1329
25
1273
29
1256
24
Python vs. JavaScript: Development Environments and Tools
Apr 26, 2025 am 12:09 AM
Both Python and JavaScript's choices in development environments are important. 1) Python's development environment includes PyCharm, JupyterNotebook and Anaconda, which are suitable for data science and rapid prototyping. 2) The development environment of JavaScript includes Node.js, VSCode and Webpack, which are suitable for front-end and back-end development. Choosing the right tools according to project needs can improve development efficiency and project success rate.
What happens if session_start() is called multiple times?
Apr 25, 2025 am 12:06 AM
Multiple calls to session_start() will result in warning messages and possible data overwrites. 1) PHP will issue a warning, prompting that the session has been started. 2) It may cause unexpected overwriting of session data. 3) Use session_status() to check the session status to avoid repeated calls.
Redis: Understanding Its Architecture and Purpose
Apr 26, 2025 am 12:11 AM
Redis is a memory data structure storage system, mainly used as a database, cache and message broker. Its core features include single-threaded model, I/O multiplexing, persistence mechanism, replication and clustering functions. Redis is commonly used in practical applications for caching, session storage, and message queues. It can significantly improve its performance by selecting the right data structure, using pipelines and transactions, and monitoring and tuning.
Composer: Aiding PHP Development Through AI
Apr 29, 2025 am 12:27 AM
AI can help optimize the use of Composer. Specific methods include: 1. Dependency management optimization: AI analyzes dependencies, recommends the best version combination, and reduces conflicts. 2. Automated code generation: AI generates composer.json files that conform to best practices. 3. Improve code quality: AI detects potential problems, provides optimization suggestions, and improves code quality. These methods are implemented through machine learning and natural language processing technologies to help developers improve efficiency and code quality.
What is the significance of the session_start() function?
May 03, 2025 am 12:18 AM
session_start()iscrucialinPHPformanagingusersessions.1)Itinitiatesanewsessionifnoneexists,2)resumesanexistingsession,and3)setsasessioncookieforcontinuityacrossrequests,enablingapplicationslikeuserauthenticationandpersonalizedcontent.
The Future of Python and JavaScript: Trends and Predictions
Apr 27, 2025 am 12:21 AM
The future trends of Python and JavaScript include: 1. Python will consolidate its position in the fields of scientific computing and AI, 2. JavaScript will promote the development of web technology, 3. Cross-platform development will become a hot topic, and 4. Performance optimization will be the focus. Both will continue to expand application scenarios in their respective fields and make more breakthroughs in performance.
Quantitative Exchange Ranking 2025 Top 10 Recommendations for Digital Currency Quantitative Trading APPs
Apr 30, 2025 pm 07:24 PM
The built-in quantization tools on the exchange include: 1. Binance: Provides Binance Futures quantitative module, low handling fees, and supports AI-assisted transactions. 2. OKX (Ouyi): Supports multi-account management and intelligent order routing, and provides institutional-level risk control. The independent quantitative strategy platforms include: 3. 3Commas: drag-and-drop strategy generator, suitable for multi-platform hedging arbitrage. 4. Quadency: Professional-level algorithm strategy library, supporting customized risk thresholds. 5. Pionex: Built-in 16 preset strategy, low transaction fee. Vertical domain tools include: 6. Cryptohopper: cloud-based quantitative platform, supporting 150 technical indicators. 7. Bitsgap:
H5: Key Improvements in HTML5
Apr 28, 2025 am 12:26 AM
HTML5 brings five key improvements: 1. Semantic tags improve code clarity and SEO effects; 2. Multimedia support simplifies video and audio embedding; 3. Form enhancement simplifies verification; 4. Offline and local storage improves user experience; 5. Canvas and graphics functions enhance the visualization of web pages.
