Troubleshooting Networks
Methodology
Tools
Wireless Issues
Cable Issues
Network Issues
Security Issues
WAN Issues
Loss of Internet Connectivity
computer needs a legitimate IP address, subnet mask, default gateway, and DNS address
1. Identify the Problem
Not what user reported
full impact of problem
Gather Info
Duplicate Problem
Log into users comp
Log in under different user
Determine if anything has changed
Ask users about any changes to ntwk
Avoid the word "you"/Accusatory tone
Check documentation
Approach Multiple Problems Individually
If problem is complex, isolate single issues and look for root cause
2. Establish Theory of Probable Cause
3. Test Theory
Without changing anything or risking repercussions
4. Establish Plan of Actions
Identify potential effects
5.Implement Solution
6. Verify Full System Functionality
On site
Identify Symptoms first hand
Remote
Question Users
Close-ended ?s for experienced users
Open-ended ?s for novice users
Question the Obvious
Start with most probable cause
Consider Multiple Approaches
Top-to-bottom OSI Model Approach
Bottom-to-top OSI Model Approach
Divide and Conquer
As you gather information for troubleshooting, a general sense of where the problem lies should manifest. Place this likely cause at the appropriate layer of the OSI model and begin to test the theory and related theories at that layer.
Theory Wrong?
Escalate the Problem
Establish New Theory
Return to step 2
Inform other parties for guidance
Pass job off to another authority who has control over device/issue
Escalate as Necessary
If you are under qualified or if it falls under someone else's duty
Document Actions and results
Try one solution at a time to isolate results
Test solution by trying to recreate issue
Ensure system works beyond initial issue
Implement Preventative Measures
Document Final Issue/Solution
Software Tools
Hardware Tools
COMMAND Tools
ipconfig and ifconfig
ping
test the connection between two nodes
Uses ICMP
Can test availability of Sites using FQDNs(Google.com)
Use to test DNS
Unknown host
This message means, “I don’t know the IP address!” You probably specified an invalid/unused DNS name.
Destination host unreachable
This message means, “I can’t get to that IP address.” In this case, you should check for possible routing problems—for example, have you specified a default gateway?
ping –t
tells ping to run indefinately
ping -6
use IPv6
tracert
traces the route between two hosts
pathping
ombines the functions of ping and tracert and adds some additional functions.
netstat
enables a network tech to examine network statistics about a system
Listening ports or established connections
nbtstat
enables a network tech to check information about the NetBIOS name
nslookup
provides a command-line utility for diagnosing DNS problems
arp
helps diagnose problems associated with the Address Resolution Protocol (ARP). CompTIA refers to the output of the arp command as the MAC address lookup table, while most folks would just call it the arp cache or arp table.
Software/Web Tools
Protocol Analyzer
Lets you look at protocols running at different levels of network
helps you determine slowdowns on a network by giving you an idea of excess or unexpected traffic
Wifi Analyzer
documents all existing wireless networks in the area
Handheld tool or software on laptop
Speed Test Sites
ipconfig The ipconfig command is used i*n Windows to display the IP address information of the system.
• ipconfig /all Displays all TCP/IP settings and the MAC address
• ipconfig /displaydns Displays the DNS resolver cache
• ipconfig /flushdns Clears out the DNS resolver cache
Looking Glass Sites
Runs diag from outside of the network
ping, traceroute, etc
Line Testers
check the integrity of telephone wiring
Light Meter
Check fiber for dust, poor connections, and light leakage
Tone Locators/Probes
AKA Fox and Hound
Tone gen emits signal
Tone node picks up signal
Cable Testers
AKA continuity testers
Test for shorts, broken/shorts
time-domain reflectometer (TDR)
Tells length of cable and where break/short is
Optical Time domain reflectormeter(OTDR) for fiber
Multimeters
measures resistance/ohms
good cable will have 0 ohms
Certifiers
will report speed and duplex settings
Signal Issues
Interference
(RMI) Radio frequency interference
Scan for RF sources using scanner/analyzer
Measure in (SNR) Singal to Noise Ratio
Overlapping Channels
Aim for 1, 6, 11
Consecutive channels have overlap
Mismatched Channels
Client set to different ch than WAP
Rare due to auto ch
Overworked WAP
Configuration
SSID/ Connectivity Mistakes
Power Levels
Easy to adjust,
most WAP are set at low power by default
Open Networks
- avoid accidentally logging into open ntwk
- provide security on open ntwk with VPN or HTTPS
Rogue Access Point
Access point with same SSID
Evil twin- set up as a trap to catch users
Wrong Antenna Type
Omnidirectional usually work fine but not always
Incompatibilities
Watch for incompatible bands
Some routers will only do 2.4 or 5.0 at any one time
Wrong Encryption
- Wrong encryption configured
- Entered incorrect passowrd
Bounce
signal sent through multiple paths
Use WAP with multiple antennas
(MIMO) multiple in/multiple out
enables devices to make multiple simultaneous connections called streams.
Channel bonding to increase throughout
AP Placement
Antenna Placement
AP Configuration
Config channel and frequency of WAP
2.4 v 5
1,6,11
Thin/Thick Client
Thick - Access WAP directly with own interface
Thin - Configured by wireless controller
Lightweight Access Point Protocol (LWAPP)
Enables interoperability
Most WAPs will accept commands from any wireless controller
Environmental Factors
150 or 300 ft is in ideal situations
Watch out for dead spots caused by concrete walls, metal and RF blocking window film
Relocate WAPs to fix
ifconfig The ifconfig command displays or sets settings on a network card on a UNIX/Linux/OS X system.
• ifconfig Displays the network card and IP settings
• ifconfig eth0 up Enables the first Ethernet card
• ifconfig eth0 down Disables the Ethernet card
Cable Failures
User Error
Crosstalk
electrical signal bleeds from one wire pair to another
poor crimping
Near-End Crosstalk (NEXT)
connected on the same end of the cable as the end emanating the signal
listens on the other three pairs and measures the amount of interference.
Far-End Crosstalk (FEXT)
sending the signal down one pair of wires, but this time listening on the other three pairs on the far end of the connection
EMI/RFI
EMI and RFI can disrupt signaling on a copper cable
Distance Limitation
100-meter distance limitation of UTP-based networks is inadequate for networks covering large buildings or campuses
Attenuation/dB Loss
weakening of a signal as it travels long distances
Bad Wiring/Connector
Split Pair
signal from any of the pairs in the same cable interfering with another pair
TX/RX Reversed
Make sure you have wired correct standards
568A/B
Incorrect VLAN Assignment
Make sure correct VLAN are assigned to correct ports
Cable Placement
Plug stuff in right
Fiber Issues :
Wavelength Mismatch
signal might be 1310nm but the switch might be expecting 1530nm
Connector Issues
Dirty connector
Slight mismatch in core or cladding can lead to major loss
Bend Radius
bend a fiber-optic cable too much, you get light leakage
Distance Limitations
check the coupler if one is used to extend a cable run
They are plastic and easily broken
SFP/GBIC Transceiver Problems-Cable Mismatch
Watch our for multimode vs single mode mismatch
Just because a connector fits does not mean that it will work
Incorrect Termination
Poor crimping
Straight-through/crossover
Open
Wire doesn't connect from end to the other
Short
Wire connects to another wire in the cable
Incorrect IP Configuration/Gateway
Go into the network configuration for the device and put in correct numbers
Broadcast Storms
result of one or more devices sending a nonstop flurry of broadcast frames on the network. Every comp on broadcast domain can't connect to ntwk
Unplug devices until you find the one flooding ntwk
Try packet analyzer
Swtiching Loops
when you connect and configure multiple switches together in such a way that causes a circular path to appear
Spanning Tree Protocol makes this rare
Duplicate IP
No two computers can have the same IP address on a broadcast domain
Speed Mismatch
link will not come up
Duplex Mismatch
link will come up but the connection will be erratic
End-to-end connectivity
Connecting users with essential resources within a smaller network, such as a LAN or a private WAN
Make sure proper ports are open, make sure users have right permissions nd ACL are setup correctly
Hardware Failure
Misconfigured DHCP
Miconfig host - cause widespread problems
Misconfig host - cause local problems
Misconfigured DNS
ping a file server by IP address but not by name, this points to DNS issues
Incorrect Interface/Misconfigured Interface
NAT rules take precedence over an appliance’s routing table entries.
Interface Errors
Patch cable could be crossover
incorrect termination
Simultaneous Wired/Wireless Connections
NIC cannot use wireless/wired connections at same time
Must configure one as default
Discovering Neighboring Devices/Nodes
If comp fails to discover devices, this might be the dhcp or dns
Power Failure/Anomalies
Get UPS to avoid issues
MTU
Maximum Transmission Unit
Ethernet packet is 1500 bytes
DSL carriers MTU 1400 bytes
MTU Black Holes
Path MTU Discovery (PMTU), determine the best MTU setting automatically,
PMTU runs under ICMP
Most routers have firewalls that block ICMP
Missing IP Routes
access control list (ACL) might block or allow access to network resources for nodes that shouldn’t have it.
NIC Teaming Misconfiguration
Link Aggregation Control Protocol (LACP)
Two NICS can be active or passive
Passive listens and Active initiaites
Passive passive will not work
Multicast vs. Broadcast
Misconfigured Firewall
Malware
Any software designed to do something the user doesn't want it to
Virus
2 Jobs: replicate and to activate
to replicate and to activate.
does not replicate across networks.
needs human action to spread.
Worm
Replicate over ntwk
do not need host files to infect
Macro
exploits application macros to replicate and activate
Trojan Horse
Appears to be friendly software like poker or screensaver
Trojan horses do not replicate
Rootkit
takes advantage of low-level operating system functions to hide
Adware
monitors the types of websites you frequent and uses that information to generate targeted advertisements
Spyware
program that sends information about your system or your actions over the Internet
Complete Uneeded Running Services
Unpatched Firmware/OS
Test patch before rolling out to entire ntwk
If legacy sys cannot be updated, put them behind some firewall
User Issues
Authentication Issues
Trusted Users
account that has been granted admin rights
Untrusted Users
account that has been granted no administrative powers.
AAA
ARP Issues
ARP enables any device at any time to announce its MAC address without first getting a request
device can just declare itself to be a “router.”
Banner Grabbing
probe a host’s open ports to learn details about running services
Mal user can send invalid request to port 80 and learn about computer's software based on error message
OUI
organizationally unique identifier
By issuing certain ICMP messages malicious user can collect all of the OUI numbers nodes attached to a network
Can then lookup manufactures for vulnerabilities
Domain/Local Group Configurations
Groups help admin avoid assigning improper access to users
Physical Issues
Interface Errors
Check all cabling between client and demark before calling ISP
Check NIC by running loopback test with loopback plug
Configuration Issues
Split Horizon
When a router learns a route through 1 Interface, it will not communicate that route on the same interface
Router Configurations
Router Protocols
specify the wrong routing protocol or misconfigure the right routing protocol.
ACLS
Include addresses to block that shouldn’t be blocked or allow access that shouldn’t have it.
Missing Routes
misconfigured router as a default gateway is either not able to get packets out or not able to get packets in
tracert/traceroute
Run traceroute to your default gateway. If that fails, you know you have a local issue and can potentially do something about it.
CSU/DSU
Copper Line Drivers/ Repeaters
Line drivers enabled installers to avoid using fiber
Company Security Policy
Throttling policy
Blocking policy
-a
ALL connections and listening ports
-b
exe involved in creating each connection of listening port
-e
ETHERNET Stats
-f
Fully Qualified Domain Names (FQDN) for foreign addresses
-n
Addresses and ports in NUMERICAL form
-p protocol
Shows connections for protocol specified
-o
OWNING process ID associated with each connection
-r/route print
ROUTING table
-s
Per-protocol Stats
-t
Current Connection offload State
-interval
Redisplays selected stats, pausing interval seconds between each display
-a
lists remote machine's name table given it's name
-A
lists temote machine's name table given its IP address
-c
lists NBT's cache of remote machine names and their IP addreses
-n
lists local NetBIOS names
-R
Purges and reloads the remote cache name table
Bandwidth Saturation
Too many devices on single band (2.4 or 5)
Signal Loss
lack enough signal power:
1.get closer to the WAP
- avoid dead spots
3.turn up the power
4.use a better antenna
5.upgrade to a newer 802.11 version (such as 802.11n or 802.11ac) with features that enable power to be used more efficiently.
Device Saturation
Too many devices on WAP
Place extra WAP in high traffic areas
Omnidirectional
Radiates outward in all directions from WAP
Unidirectional
Directs signal in specific direction
Patch
Flat plate shaped antennas
Generates half sphere beam
Always on walls
Untested Updates
Always test updates before pushing them to entire ntwk
Wrong SSID
Take care to log onto correct ntwk
click to edit
Tight Control of User Accounts
Unauthorized access
means a person does something beyond his or her authority.
Improper access
occurs when a user who shouldn’t have access gains access through some means.
Malicious users
Watch out for default user access
Failing to point the switch to the correct RADIUS/TACACS+ server
You need to give the switch the right IP address. It’s a simple issue, but one that often happens.
Improperly configuring the correct authentication method for the switch
If you configure the switch for EAP-PEAP and the server is expecting EAP-TLS, it won’t work properly. If you want to use a certificate-based authentication, you’ll need a valid certificate that the server can use.
Failing to give the switch proper security policies on the server
In this case, the switch won’t be allowed to do its job.
Interference
New installations can cause EMI
DNS Issues
1) ISP DNS Server can fail
2) ISP DNS servers use "DNS Helpers" that redirect you to ads when the wrong URL is entered.
Use fast public DNS
Google: 8.8.8.8 as primary or backup DNS
ICMP Issues
Ping of Death
malformed ping packet that was too large for the protocol to handle and would therefore crash the software system of the computer
Unreachable Default Gateway
Ping returns issue
Trusted users have access to resources that they shouldn't
Untrusted users made their way onto the system