Latency
Latency
The delay before an operation or packet completes.
In everyday business: Users experience high latency as slowness even when headline bandwidth looks good.
ISCSI
iSCSI Multipath and SAN Troubleshooting Playbook
Trace iSCSI from host initiator to target, separate network latency from storage latency, and build real path redundancy.
Book 4 | Chapter 4
Chapter opening
Chapter 4 moves from the course overview into iscsi multipath and san troubleshooting. The aim is not to memorise product menus or commands. The reader should understand what problem the technology solves, how it fits into the wider storage technical textbook, what normal operation looks like, and how to prove the result in the field.
The previous chapter introduced nas deployment, shares, permissions and capacity playbook. This chapter continues the sequence by focusing on iscsi multipath and san troubleshooting playbook, so the reader can connect the architecture to a more specific operational problem.
Learning objectives
- Explain the main purpose of iSCSI Multipath and SAN Troubleshooting in plain language.
- Distinguish capacity, performance, resilience and backup instead of treating them as the same requirement.
- Use evidence to separate a design problem from a configuration or physical fault.
- Describe the business impact of a failure and the evidence required for handover.
Why this matters in a real company
Storage is where company files, databases and virtual machines live. The business impact is not only capacity: performance, resilience, backup and recovery determine whether data remains usable when something fails.
Course: Storage Technical Textbook
PLAIN ENGLISH
Technical terms used in this chapter
You do not need to memorise the jargon first. Understand what each term does and why somebody running a company would care about it.
iSCSI
Internet Small Computer Systems Interface
A protocol that carries block-storage commands over IP networks.
In everyday business: A server can use remote storage almost as if it were a local disk.
VLAN
Virtual LAN
A logical Layer 2 network that separates devices even when they use the same physical switches.
In everyday business: Staff, cameras and guest WiFi can be kept apart without buying a separate switch for each.
PAT
Port Address Translation
A form of NAT that lets many internal connections share one public address by tracking transport ports.
In everyday business: Hundreds of users can browse the Internet through one public IPv4 address.
SAN
Storage Area Network
A dedicated storage network that presents block storage to servers.
In everyday business: Multiple servers can use centrally managed storage without treating it like a normal file share.
BUSINESS TRANSLATION
What this means outside the server room
This topic affects where company data lives, how fast applications respond and whether the business can continue after hardware or data problems.
RAID
What it means: RAID combines several drives so the storage system can keep operating through certain drive failures, depending on the RAID level.
In normal business language: Think of RAID as keeping the business open when one filing cabinet drawer breaks. It helps availability, but it does not protect you if somebody deletes the wrong folder, malware encrypts the files, or the whole building is lost.
Why the decision matters: Use RAID to reduce downtime from disk failure, and use backup for recovery from deletion, corruption, ransomware and larger disasters.
VLAN
What it means: A VLAN creates a separate logical network even when devices share the same physical switches.
In normal business language: Think of one office building with separate departments using the same corridors but different access-controlled areas. Staff, cameras, voice and guests can share the same switching hardware without all being in one open network.
Why the decision matters: Use VLANs to separate traffic for security, performance and easier troubleshooting without buying a separate physical switch for every department.
Virtual Machine
What it means: A virtual machine is a software-defined computer with its own operating system, memory, CPU allocation, disks and network interfaces.
In normal business language: It behaves like a separate server even though several VMs may share one physical machine.
Why the decision matters: VMs are useful for separating roles such as domain control, file sharing and applications while keeping management and recovery flexible.
Backup
What it means: A backup is an independent recoverable copy of data or system state.
In normal business language: Think of it as having a duplicate of important company records stored safely away from the live filing cabinet.
Why the decision matters: A backup only counts if it can be restored. Test recovery, not just backup completion.
NAT
What it means: NAT changes IP addresses as traffic crosses a network boundary.
In normal business language: It is similar to a company switchboard: many internal extensions can share one public number.
Why the decision matters: NAT helps with address use and publishing services, but it is not a substitute for firewall security.
NAS
What it means: NAS is network file storage, usually shared with protocols such as SMB or NFS.
In normal business language: It is a central company filing cabinet that many staff can access over the network.
Why the decision matters: NAS is a good fit for shared files, backups and general file services, provided permissions, backup and capacity are planned.
4.1 Two NICs are not automatically two paths
True multipath design requires independent usable network paths and host multipath software that understands them. If both NICs depend on one switch, one VLAN failure or one storage port, the design still contains a shared failure point.
Record the path
- Host initiator IQN.
- Host NIC/IP for each path.
- Storage target IQN.
- Target portal IPs.
- VLAN and switch port for each path.
- LUN identifier.
- Multipath policy.
MTU
If jumbo frames are used, every hop in the storage path must support the selected MTU. Partial jumbo configuration can create intermittent or confusing behaviour. Test end to end with a packet size that proves the intended MTU rather than checking only the server NIC setting.
Latency isolation
NetApp documents cases where iSCSI workloads show low array-side latency but high network delay. That is why the installer should check host, network and storage metrics independently before blaming disks.
Network checks
show interfaces counters errors
show interfaces <port>
show mac address-table
Failover test
- Confirm all expected paths are active.
- Generate controlled storage I/O.
- Disable one approved network path.
- Confirm the host continues I/O through another path.
- Restore and confirm the path rejoins.
What does not work
- Using ordinary NIC teaming where the storage vendor requires multipath I/O.
- Setting jumbo frames on only host and array.
- Allowing iSCSI to share an oversubscribed user uplink without measuring contention.
- Changing path policy while troubleshooting an unrelated array issue.
4.2 Session vs path
An iSCSI target can be reachable while multipathing is still incomplete. Confirm every intended session/path is established and that the host multipath layer presents one logical device rather than duplicate unmanaged disks.
Switch design
Keep storage VLAN configuration consistent across both fabrics. Check flow control or data-centre features only if the storage and switch vendors require them. Do not enable features from another vendor's design guide without confirming compatibility.
Packet loss
Storage protocols are sensitive to retransmission and latency. Watch interface errors, drops and oversubscription. If the array reports low storage latency while the application is slow, the host and network path deserve equal attention.
Maintenance test
Fail one path at a time during an approved window and watch application I/O, multipath state and recovery. A design that has never survived a controlled path failure is not proven redundant.
4.3 Discovery, login and presentation are different stages
An initiator can discover a target portal without successfully logging in, and it can log in without receiving the intended LUN. Troubleshoot the sequence separately: IP reachability, target discovery, authentication, session establishment, LUN mapping, multipath presentation and filesystem/application use.
Authentication
CHAP credentials, if used, belong to the iSCSI session. A successful ping does not prove iSCSI authentication will work. Keep initiator and target identity records so access can be audited.
Path ownership
Some arrays have preferred paths or controllers. Multipath software can use active/active, active/passive or vendor-specific behaviour. Follow the array's host integration guide rather than forcing a generic round-robin policy.
Storage VLAN routing
Many iSCSI designs keep storage networks non-routed to reduce complexity and exposure. If routing is intentionally used, account for MTU, latency, security policy and multipath design across every routed hop.
Fault example
If one path repeatedly drops while the other is stable, compare switch errors, optics/cabling, host NIC logs and target-port counters for that path only. Do not reset the entire storage system before isolating the failing leg.
WORKED BUSINESS SCENARIO
Scenario: storage is online, but the business says it is slow
A storage system has free capacity and all disks show healthy, yet accounting reports are taking much longer and VM backups are overlapping into business hours.
What the chapter teaches us: Capacity and health LEDs do not prove performance. The investigation must separate application delay, host queues, network path, controller/cache behaviour and media performance.
Think it through
- Which measurements distinguish latency from throughput?
- Could the network be the bottleneck even when the array is healthy?
- What baseline would help prove whether the problem is new?
PRACTICAL WORK
Hands-on lab
Document a lab storage path from application to virtual or physical disk. Record usable capacity, RAID layout, network path if any, and backup destination.
Troubleshooting exercise
Storage is slow. Separate application, host queue, network, controller or cache and media metrics before replacing drives.
Chapter field checklist
- I can explain the subject without relying only on acronyms.
- I can draw or describe the main traffic, storage, power or service path.
- I know what normal operation should look like.
- I know which logs, counters or test results prove the result.
- I can explain the business impact if this component fails.
- I would document the final configuration and evidence at handover.
Chapter summary
iSCSI Multipath and SAN Troubleshooting Playbook should now be understood as a business service with a technical implementation, not simply a collection of commands or product names. The important habit is to know the purpose, understand the dependencies, measure the result and document what was proven.
END OF CHAPTER
Review questions
- Explain the subject in plain language to a business owner.
- List the main dependencies that must be healthy before this service can work.
- Describe one realistic failure and the first three pieces of evidence you would collect.
- Explain what should be documented at handover.
A good answer should explain the reason, not only repeat a product name or command.
Technical references
Use the current project specification and the exact product documentation for the installed equipment. These references support the technical principles used in this guide.