题目
设计基于NIO的高性能日志系统，实现异步写入与日志滚动
信息
- 类型：问答
- 难度：⭐⭐⭐
考点
AsynchronousFileChannel,缓冲区管理,日志滚动策略,异常处理,并发控制
快速回答
实现高性能NIO日志系统的核心要点：
- 使用
AsynchronousFileChannel实现非阻塞写入 - 采用双缓冲区机制避免写入阻塞
- 实现基于文件大小/时间的日志滚动策略
- 通过
CompletionHandler处理异步操作结果 - 使用原子变量保证线程安全
- 添加背压机制防止内存溢出
原理说明
高性能日志系统需要解决传统IO的阻塞问题。通过AsynchronousFileChannel实现真正的异步写入，结合双缓冲区设计：前台缓冲区接收日志，后台缓冲区异步写入文件。当日志文件达到阈值时触发滚动（创建新文件），同时确保写入操作的原子性和异常恢复能力。
代码实现
import java.io.IOException;
import java.nio.ByteBuffer;
import java.nio.channels.AsynchronousFileChannel;
import java.nio.channels.CompletionHandler;
import java.nio.file.*;
import java.util.concurrent.atomic.AtomicBoolean;
import java.util.concurrent.atomic.AtomicLong;
import java.util.concurrent.atomic.AtomicReference;
public class NioLogger {
private static final int BUFFER_SIZE = 4 * 1024 * 1024; // 4MB缓冲区
private final AtomicReference<ByteBuffer> frontBuffer = new AtomicReference<>(ByteBuffer.allocate(BUFFER_SIZE));
private final AtomicReference<ByteBuffer> backBuffer = new AtomicReference<>(ByteBuffer.allocate(BUFFER_SIZE));
private final AtomicBoolean isWriting = new AtomicBoolean(false);
private final AtomicLong currentPosition = new AtomicLong(0);
private final AtomicLong currentFileSize = new AtomicLong(0);
private final Path logDir;
private final String logPrefix;
private final long maxFileSize;
private AsynchronousFileChannel currentChannel;
public NioLogger(String dir, String prefix, long maxSize) throws IOException {
this.logDir = Paths.get(dir);
this.logPrefix = prefix;
this.maxFileSize = maxSize;
Files.createDirectories(logDir);
rollFile(); // 初始化日志文件
}
public void log(String message) {
byte[] bytes = (message + "\n").getBytes();
while (true) {
ByteBuffer buffer = frontBuffer.get();
if (buffer.remaining() >= bytes.length) {
buffer.put(bytes);
return;
} else if (isWriting.compareAndSet(false, true)) {
swapBuffers();
scheduleWrite();
}
// 背压：缓冲区满时短暂阻塞
try { Thread.sleep(1); } catch (InterruptedException e) { /* 处理中断 */ }
}
}
private void swapBuffers() {
ByteBuffer tmp = frontBuffer.get();
frontBuffer.set(backBuffer.getAndSet(tmp));
tmp.flip(); // 准备读取
}
private void scheduleWrite() {
ByteBuffer buffer = backBuffer.get();
long writePosition = currentPosition.getAndAdd(buffer.remaining());
currentChannel.write(buffer, writePosition, buffer, new CompletionHandler<Integer, ByteBuffer>() {
@Override
public void completed(Integer result, ByteBuffer attachment) {
attachment.clear(); // 重置缓冲区
currentFileSize.addAndGet(result);
isWriting.set(false);
// 检查日志滚动条件
if (currentFileSize.get() >= maxFileSize) {
try { rollFile(); }
catch (IOException e) { handleError(e); }
}
}
@Override
public void failed(Throwable exc, ByteBuffer attachment) {
handleError(new IOException("写入失败", exc));
attachment.clear();
isWriting.set(false);
}
});
}
private synchronized void rollFile() throws IOException {
if (currentChannel != null) currentChannel.close();
String fileName = logPrefix + "_" + System.currentTimeMillis() + ".log";
Path newPath = logDir.resolve(fileName);
currentChannel = AsynchronousFileChannel.open(
newPath,
StandardOpenOption.CREATE,
StandardOpenOption.WRITE
);
currentPosition.set(0);
currentFileSize.set(0);
System.out.println("Rolled to new file: " + fileName);
}
private void handleError(Exception e) {
// 错误处理策略：重试/告警/备用存储
System.err.println("日志错误: " + e.getMessage());
try { rollFile(); } // 尝试恢复
catch (IOException ex) { ex.printStackTrace(); }
}
public void shutdown() throws IOException {
if (currentChannel != null) {
currentChannel.close();
}
}
}最佳实践
- 缓冲区设计：双缓冲区（4MB+）平衡内存使用和IO效率
- 背压机制：缓冲区满时轻度阻塞生产者线程，避免OOM
- 原子操作：使用
AtomicReference保证缓冲区交换的原子性 - 异常恢复：写入失败时自动滚动新文件，结合告警机制
- 资源清理：显式关闭文件通道防止资源泄漏
常见错误
- 缓冲区竞争：未使用原子操作导致数据损坏
- 丢失日志：关闭应用时未刷新缓冲区剩余数据
- 滚动死锁：文件滚动时未暂停写入操作
- 内存泄漏：未重用ByteBuffer导致频繁GC
- 异常忽略：未处理
CompletionHandler.failed()造成静默失败
扩展知识
- 高性能替代方案：Memory-Mapped Files（
FileChannel.map）减少内核/用户空间拷贝 - 结构化日志：结合Protocol Buffers/FlatBuffers优化序列化效率
- 分布式场景：通过
Selector实现日志批量网络传输 - OS特性利用：Linux的
O_DIRECT标志绕过页缓存（需对齐内存） - 监控指标：跟踪缓冲区交换频率/写入延迟/滚动次数等关键指标
