Many things can impact a server's file-serving performance. Fullness of the file-system is but one of many things that can contribute.

- Raw disk throughput. If the numbers of I/Os being thrown at your disks exceeds their ability to keep up, it'll get slow.
- Disk I/O patterns. Some disks behave better with massively random I/O than others. SATA, for instance, doesn't perform as well with massively-random I/O as SAS or SCSI drives.
- Disk controller resource exhaustion. Whatever you're using for RAID (presuming you are, and this isn't just a single disk) has its own resources. If you're using a parity RAID, it's controller CPU that limits how fast you can commit data to disk. Also, most hardware controllers have their own onboard cache. This is used for many things, but includes reordering writes for improved efficiency. If I/O gets too random, your RAID card may not be able to optimize as well.
- File-cache memory resources. File-servers perform best when they can fully cache 100% of the open files in memory. This allows them to accept writes from clients and reorder commits to disk in such a way as to make them more efficient. If you can't fit your entire open file set in memory, it'll have to go direct to disk for those I/Os and you'll lose this performance enhancement.
- Client-local memory resources. Through the use of OpLocks, clients can cache open files locally on themselves. Once more than one client opens the same file, the server tells the client to flush its cache, and this goes away. However, for some workloads it can be a real savings. If the client doesn't have enough file-cache space to cache open files, performance can degrade noticeably when opening files exclusively.
- File-system fragmentation. A massively fragmented file-system by its very nature induces a massively random I/O pattern on the disk subsystem. If that sub-system can't tolerate that sort of I/O pattern, things get real slow.
- User-generated I/O patterns. If your users are working on millions of office documents (generally under 2MB in size) your access patterns are going to be very random. If your users are working on large files such as video files, geospatial data, or AutoCAD files, your users will be generating a lot of sequential operations.

Some of these interrelate and many times it'll be multiple issues driving a performance problem. In general, NTFS filesystem fragmentation does have an impact. The impact is worst when doing large sequential reads from such a file-system, such as happens during a backup. The impact to general file-serving performance is not as significant for typical office-server loads since those are largely random I/O anyway; and in some cases you can even see some performance improvements with a fragmented system over a fully defragged one.

For a file-server storing a lot of AutoCAD files, NTFS fragmentation will be perceptible to the end users. That user-generated I/O pattern is significantly sequential, and is therefore vulnerable to degradation by fragmentation. How much it'll be really impacted is dependent upon how much RAM the server has for caching, and how fast the underlaying storage is regarding random I/O patterns. It could very well be that the underlaying storage is fast enough that end-users won't notice a volume with 60% fragmentation. Or it could cause I/O saturation with only 15% frag.

For a file-server storing a lot of plain old office files, NTFS fragmentation will not be as perceptible to end users. That user I/O pattern is significantly random as it is, and is minimally impacted by fragmentation. Where the problems will emerge is in the backup process, as the time to backup each GB will increase as fragmentation increases.

Which brings me to my final point. The one I/O operation that is most affected by fragmentation is sequential I/O. Most servers undergo large scale sequential I/O patterns as part of the backup process. If you're having trouble fitting your backup into your backup window, defragging can help make things go faster. Your underlaying storage systems will determine how much of an impact fragmentation can have, and your fragmentation numbers will determine how much of an impact it actually has. Know your storage.