New I/O

旧的I/O包已经使用nio重新实现过，以便充分利用这种速度提高

速度的提高来自于所使用的数据集结构更接近于操作系统执行的I/O方式：通道和缓冲器，唯一直接与通道交互的缓冲器ByteBuffer。旧的I/O库中的FileInputStream,FileOutputStream,RandomAccessFile可以用来生成FileChannel;Reader和Writer不能用于产生通道，但是在java.nio.channels.Channels类中提供了在通道中产生Reader和Writer的方法。

/*** Constructs a reader that decodes bytes from the given channel using the given decoder.*/
public static Reader newReader(ReadableByteChannel ch, CharsetDecoder dec, int minBufferCap){checkNotNull(ch, "ch");return StreamDecoder.forDecoder(ch, dec.reset(), minBufferCap);// new StreamDecoder(...);
}
public static Reader newReader(ReadableByteChannel ch, String csName){checkNotNull(csName, "csName");return newReader(ch, Charset.forName(csName).newDecoder(), -1);
}public static Writer newWriter(final WritableByteChannel ch, final CharsetEncoder enc, final int minBufferCap){checkNotNull(ch, "ch");return StreamEncoder.forEncoder(ch, enc.reset(), minBufferCap);// ..
}
public static Writer newWriter(WritableByteChannel ch, String csName){checkNotNull(csName, "csName");return newWriter(ch, Charset.forName(csName).newEncoder(), -1);}

ByteBuffer

A byte buffer is either direct or non-direct.

• Given a direct byte buffer, the Java virtual machine will make a best effort to perform native I/O operations directly upon it. That is, it will attempt to avoid copying the buffer’s content to (or from) an intermediate buffer before (or after) each invocation of one of the underlying operating system’s native I/O operations.

• The contents of direct buffers may reside outside of the normal garbage-collected heap, and so their impact upon the memory footprint of an application might not be obvious.

• It is therefore recommended that direct buffers be allocated primarily for large, long-lived buffers that are subject to the underlying system’s native I/O operations.

• In general it is best to allocate direct buffers only when they yield a measureable gain in program performance.

ByteBuffer(int mark, int pos, int lim, int cap,   // package-privatebyte[] hb, int offset){super(mark, pos, lim, cap);this.hb = hb;     		///   Non-null only for heap buffersthis.offset = offset;	///   
}
Buffer(int mark, int pos, int lim, int cap) {       // package-privateif (cap < 0)throw new IllegalArgumentException("Negative capacity: " + cap);this.capacity = cap;limit(lim);position(pos);if (mark >= 0) {if (mark > pos)throw new IllegalArgumentException("mark > position: (" + mark + " > " + pos + ")");this.mark = mark;}}

Direct Buffers

// Allocates a new direct byte buffer.
// this method typically have somewhat higher allocation and deallocation costs than non-direct buffers. 更高的分配和释放成本
public static ByteBuffer allocateDirect(int capacity) {return new DirectByteBuffer(capacity);
}// Maps a region of this channel's file directly into memory.
//Read/write: Changes made to the resulting buffer will eventually be propagated to the file; they may or may not be made visible to other programs that have mapped the same file.
//它们可能对映射了同一文件的其他程序可见，也可能不可见。
//Private: Changes made to the resulting buffer will not be propagated to the file and will not be visible to other programs that have mapped the same file; instead, they will cause private copies of the modified portions of the buffer to be created.                                  
public abstract MappedByteBuffer map(MapMode mode, long position, long size) throws IOException;

MappedByteBuffer

A direct byte buffer whose content is a memory-mapped region of a file.
For most operating systems, mapping a file into memory is more expensive than reading or writing a few tens of kilobytes of data via the usual {@link #read read} and {@link #write write} methods. From the standpoint of performance it is generally only worth mapping relatively large files into memory.

示例

将E://test目录下的大文件进行读写

//Main.java
private static void testFiles() {ExecutorService executorService = Executors.newFixedThreadPool(12);//....FileChannel fileChannel = new RandomAccessFile(file1, "rw").getChannel();for (int j = 0; j < 8; j++) {MappedByteBuffer buffer = fileChannel.map(FileChannel.MapMode.READ_WRITE, j * block, block);executorService.execute(new RWTest(file1.getName(), fileChannel, buffer, j * block, block));}}  // 文件太小}
//RWTest.java
public void run() {long start = System.currentTimeMillis();FileLock fileLock = null;while (true) {try {fileLock = fileChannel.tryLock(position, block, false);break;} catch (IOException e) {System.out.println("there is another thread here.");try {Thread.sleep(1000);} catch (InterruptedException e1) {e1.printStackTrace();}}}// do something ...int index = 0;while (buffer.hasRemaining()) {buffer.put((byte) (buffer.get(index) ^ chaos[index % 16]));//数据读写index++;}try {if (fileLock != null)fileLock.release();} catch (IOException e) {e.printStackTrace();}long stop = System.currentTimeMillis();System.out.println(String.format("%s 耗费 %d毫秒", this, stop - start));
}  

// 首次运行
Thread pool-1-thread-3-14 	position:643952840  block: 321976420    covert:aa.mkv  耗费 25074毫秒
Thread pool-1-thread-7-18 	position:1931858520  block: 321976420    covert:aa.mkv  耗费 25310毫秒
Thread pool-1-thread-8-19 	position:2253834940  block: 321976420    covert:aa.mkv  耗费 26502毫秒
Thread pool-1-thread-2-13 	position:321976420  block: 321976420    covert:aa.mkv  耗费 27924毫秒
Thread pool-1-thread-4-15 	position:965929260  block: 321976420    covert:aa.mkv  耗费 28847毫秒
Thread pool-1-thread-6-17 	position:1609882100  block: 321976420    covert:aa.mkv  耗费 28859毫秒
Thread pool-1-thread-5-16 	position:1287905680  block: 321976420    covert:aa.mkv  耗费 29086毫秒
Thread pool-1-thread-1-12 	position:0  block: 321976420    covert:aa.mkv  耗费 29309毫秒// 第二次转码，在内存中，速度极快
Thread pool-1-thread-2-13 	position:321976420  block: 321976420    covert:aa.mkv  耗费 1248毫秒
Thread pool-1-thread-7-18 	position:1931858520  block: 321976420    covert:aa.mkv  耗费 1275毫秒
Thread pool-1-thread-4-15 	position:965929260  block: 321976420    covert:aa.mkv  耗费 1285毫秒
Thread pool-1-thread-6-17 	position:1609882100  block: 321976420    covert:aa.mkv  耗费 1290毫秒
Thread pool-1-thread-5-16 	position:1287905680  block: 321976420    covert:aa.mkv  耗费 1297毫秒
Thread pool-1-thread-8-19 	position:2253834940  block: 321976420    covert:aa.mkv  耗费 1300毫秒
Thread pool-1-thread-3-14 	position:643952840  block: 321976420    covert:aa.mkv  耗费 1311毫秒
Thread pool-1-thread-1-12 	position:0  block: 321976420    covert:aa.mkv  耗费 1348毫秒

参考理解

占小狼-深入浅出MappedByteBuffer
mrguozp-MappedByteBuffer

传统的基于文件流的方式读取文件方式是系统指令调用，文件数据首先会被读取到进程的内核空间的缓冲区，而后复制到进程的用户空间，这个过程中存在两次数据拷贝；而内存映射方式读取文件的方式，也是系统指令调用，在产生缺页中断后，CPU直接从磁盘文件load数据到进程的用户空间，只有一次数据拷贝。

性能分析
从代码层面上看，从硬盘上将文件读入内存，都要经过文件系统进行数据拷贝，并且数据拷贝操作是由文件系统和硬件驱动实现的，理论上来说，拷贝数据的效率是一样的。
但是通过内存映射的方法访问硬盘上的文件，效率要比read和write系统调用高，这是为什么？

read()是系统调用，首先将文件从硬盘拷贝到内核空间的一个缓冲区，再将这些数据拷贝到用户空间，实际上进行了两次数据拷贝；
map()也是系统调用，但没有进行数据拷贝，当缺页中断发生时，直接将文件从硬盘拷贝到用户空间，只进行了一次数据拷贝。
所以，采用内存映射的读写效率要比传统的read/write性能高。

总结
MappedByteBuffer使用虚拟内存，因此分配(map)的内存大小不受JVM的-Xmx参数限制，但是也是有大小限制的。
如果当文件超出1.5G限制时，可以通过position参数重新map文件后面的内容。
MappedByteBuffer在处理大文件时的确性能很高，但也存在一些问题，如内存占用、文件关闭不确定，被其打开的文件只有在垃圾回收的才会被关闭，而且这个时间点是不确定的。