Please enable JavaScript.

Coggle requires JavaScript to display documents.

Distributed Systems & Networks, MORE ON LECTURE, MORE ON LECTURES,…

- - - - Data
    - - TCP - Data
    - - IP - TCP - Data
    - - Ethernet - IP - TCP - Data
  - - - From one sender to one receiver
    - - From one sender to all potential receivers
      - Some protocols rely on one device being able to send a message seen by all other devices at that network layer
      - Can have link layer and network layer broadcasts
    - - From one sender to any number of 'interested' receivers
    - - From one sender to nearest instance of receiver
  - - - create a new communication end point
    - - Attack a local address to a socket
    - - Announce willingness to accept connections; give queue size
    - - Block the caller until a connection attempt arrives
    - - Actively attempt to establish a connection
    - - Send some data from the connection
    - - Receive some data from the connection
    - - Release the connection
- - - - Including a number of required features
      - Encapsulates IP datagrams into Link Layer Function
  - - - For low error rate networks e.g. wired ethernet
      - It's 'connectionless' in that no signalling path is established in advance - the frames are sent, and may or may not be received by the destination
    - - e.g wireless 802.22, wifi
      - 802.11n supports block acknowledgements
    - - For long delay, unreliable links, e.g. satellite
  - - - Send a frame, wait for ACK, send next frame, etc
      - Will not get an ACK if frame is lost or damaged
    - - send multiple frames before receiving the first ACK
      - Go-back-N ARQ
        
        use a sequence tracker to label each frame in flight
        
        send so many frames of data; if an ACK is missed, retransmit from that frame
      - select-Repeat ARQ
        
        similar, but only retransmit lost frames
  - - - requires error or packet loss detection and subsequent retransmission
      - though the upper layers may have their own error detection methods as well
    - - Simplest method is parity bit
        
        is number of 1's even or odd?
        
        clearly will not reveal all errors
      - we require a more robust method
        
        Example: CRC - Cyclic Redundancy Check
        
        Result is held in the "checksum" field of the frame
        
        calculated by sender and receiver, and result compared
      - Checksums map happen at other layers too - IPv4 has a checksum, IPv6 does not
  - - - To indicate where the frame starts and ends
      - For example, to form an ethernet frame
      - uses some bandwidth to indicate the start/end of frames
    - - If FLAG occurs in the actual data, use an Escape byte
      - when receiving, strip (ignore) first Escape byte
      - If an escape byte occurs in the data, Escape that
  - - - twisted pair cable with hubs/repeaters
      - packet switched
        
        one device per switch port
        
        campus has 1 Gbit/s to the desk and 10Gbit/s backbones
        
        no connection over a medium if switch has sufficient internal bandwidth
    - - WiFi is essentially the wireless alternative to Ethernet
        
        Uses the IEEE 802.11 suite of protocols
        
        works in 2.4GHz or 5GHz range (also new 6GHz and 60GHz)
      - Has evolved over many years - 802.11b, 802.11a, 802.11g, 802.11n
      - Devices associate with a wireless access point (AP)
      - Share media, so significant contention in busy areas - therefore uses a collision avoidance scheme - frequency channels help to minimise contention
        
        Handling media contention
        
        Original Wire Ethernet - used Carrier Sense Multiple Access with Collision Detection (CSMA/CD)
        
        Sender listens to see if the media is busy - if it is busy, the sender waits
        
        When channel is free, the sender starts to talk - listen while sending and stop if collision occurs
        
        back off before retransmitting if collision detected - the pick the back off delay from an increasing set of values
        
        Used for Ethernet on single, shared media to ensure only one sender is transmitting at any time - less important in modern fully-duplex switched Ethernet networks
        
        WiFi and collision avoidance
        
        WiFI doesn't use CSMA/CD because WiFI devices can't generally send and receive at the same time
        
        There is also the 'hidden node' problem - devices see AP but not each other
        
        Instead they use CSMA/CA (CSA/ Collision Avoidance)
        
        CSMA/CA is like CSMA/CD but instead of listening into the transmission, it waits for an acknowledgement from the AP to determine if the frame was successfully sent. Faster 802.11n allows block acknowledgements
        
        Can also optionally use Request to Send /Clear to Send (RTS/CTS) signalling to the AP to improve performance
- - - - Hides routing form the layer above (transport)
      - The internet as a 'network of networks'
      - LANs at the edges, served by routers
      - routers in the core
    - - Packetising (taking Transport header and data, adding IP header)
      - Processing and Routing IP datagrams
      - Fragmenting (if necessary, where IP datagram size > link layer Max Transmission Unit - MTU)
    - - Error checking
      - Reassembly of fragments if necessary
  - - - IP packets routed towards destination at each router on the path
      - Connectionless: packet from A to B
    - - No guarantee an given sent IP datagram will be received
      - Router by default forward only on a best effort basis
    - - Usually based only on destination IP address
      - Routers maintain routing tables to make routing decisions
    - - Devices must be globally addressable to initiate IP traffic towards them
      - Private address space (RFC 1918) may be used within sites - not globally routed because by nature its ambiguous
  - - - With TCP, retransmissions are inherently handled
      - With UDP, application layer must handle retransmission
  - - - Adds IP header to create IP datagram
      - The TCP or UDP header and data become the payload
    - - Source IP Address
      - Destination IP address
      - IHL - header length (which may vary)
      - Identification field and fragment offset
      - Time to live
      - IP header checksum
    - - IP datagram may be fragmented on its path if a link with a smaller link layer MTU is encountered
        
        Ethernet typically has an MTU of 1500 bytes
        
        In IPv4 hosts and routers can fragment IP datagrams
        
        It's preferable to avoid fragmentation
        
        Identification field and fragment offset allow reassembly
        
        But it adds inefficiency, and possible security issues
    - - Used since the 1970's - since 1995 allocated via RIRs with variable length prefixes - as of February 2011 IANA still ran out of unused address space
      - Dynamic Host Configuration DHCP leases
        
        Allows IP addresses to be reused
      - Network Address Translation NAT
        
        Use private IPv4 addresses (RFC 1918) internally, global(s) externally
        
        Very popular, but comes with an architectural price
        
        Used in home networks with one IPv4 address your ISP assigns to you
  - - - The subnet network address (the first IP in the range)
      - The subnet prefix length
      - The subnet mask (the fixed prefix bits for all hosts in the subnet) / network mask / netmask
      - The subnet IP broadcast address, for protocols using IP broadcasts
    - - Allows very specific address allocations (network prefixes) and routing based on those prefixes
      - A site or ISP requesting address soccer gets exactly what it needs
    - - A /24 has 256 addresses (-3 usable)
      - Not all can be used for hosts
        
        reserved network address (first address)
        
        subnet IP broadcast address (last address)
        
        One IP address is also required for the router - typically first or last available
    - - if the destination IP address is within the same local subnet (ethernet LAN), the sender should ARP for the MAC address of the destination IP address - to send the IP packet directly
      - But if the destination IP address is not in the same subnet, the sender should ARP for the Ethernet address of its default router - to punt the onward delivery to the router
      - Which is why being able to use the net mask to determine this is very important
    - - If too short
        
        Host may think a remote host is local (ARP locally)
        
        So won't get a response
      - If too long
        
        Host may think a local host is remote (send to router)
        
        hough the router in this case can redirect
- - - - ECS has around 3000 wired ethernet points
      - currently has 1Gbit/s
      - We also have widespread 802.11n wireless
    - - Mainly wireless for convenience
      - but with increasing use of power line ethernet
    - - each host typically connects to a port on a switch
        
        Ethernet Switches
        
        Receive Ethernet frames and make a decision whether to forwards the frame, and if so, on which port
        
        Provides 'smart' forwarding
        
        It learns Ethernet addresses of host(s) seen on each switch interface/port
        
        Host generally only see packets sent to them, or broadcast packets
        
        Can store and forward Ethernet frames - so can check for errors (CRC check) before forwarding
        
        Some switches support more advanced functions - e.g. VLANs, QoS,...
      - today desktops typically 1Gbit/s Ethernet
      - Servers and backbone on 10 Gbit/s Ethernet
  - - - 48-bit source and destination MAC addresses
      - 802.1Q tag for optional VLAN ID and frame priority
  - - - Wired: IEEE 802.3 family
      - Wireless: IEEE 802.11 family
  - - - The special ethernet broadcast address is ff:ff:ff:ff:ff:ff - sender asks "who has this IP address?"
      - Seen by all hosts in the same ethernet LAN - the host with the target IP address responds
      - Sender can then use result to send ethernet frame - will typically cache the result for a period of time (e.g. 60s)
    - - Hosts will cache ARP responses to avoid repeated lookups - it can be useful to check the cache when troubleshooting
    - - Is it secure?
        
        ARP is potentially open to spoofing
        
        Note that a host can change its MAC address
      - Change of IP or MAC address
        
        Send a 'gratuitous ARP'
        
        enables a quick update of the correct information
      - ARP probe
        
        to detect IP address clashes
  - - - In a 20,000 host campus network, that's a lot of broadcasts
      - wireless APs will also be plugged in
      - And ARP isn't the only protocol that does broadcasts
    - - A LAN will use a certain range of network layer IP addresses
      - If we wish to connect two such LANs together, we need to use an IP (network layer) router
      - The router then forwards IP packets between the LANs - routers do not forward ethernet (link layer) broadcasts; they handle IP
      - The router will typically advertise to other routers the reachability of the IP address range it directly serves
        
        Using an appropriate routing protocol to share the routes
        
        Routers on a campus network thus learn where to forward IP packets
      - The Ethernet destination address
        
        If the destination is in the same LAN, then the destination MAC address is the receiver's Ethernet address
        
        A host will send direct to another host, or to its 'default' IP router
        
        But what if the destination is in a different LAN, or another remote network
        
        Ethernet addresses and routers
        
        IP routers act as network layer gateways between individual link layer ethernet LANs
        
        We are creating disjoint link layer networks, connected by network layer routes
        
        Each LAN has a block of IP addresses to use
        
        The 'default' router handles traffic sent to IP addresses which are not on the local LAN
        
        How does the send know if destination IP is local or not?
        
        IP subnets and IP routers
        
        An IP router sits at the edge of an Ethernet LAN
        
        In a campus, there may typically be 20-500 hosts per LAN
        
        Because they do not forward link layer broadcasts, all ethernet broadcasts are contained to the IP subnets 'behind' each router
        
        The administrators assign an IP subnet to each LAN
        
        An IP address plan, determines which LANs use which IP address ranges (prefixes)
        
        Similarly home network is a single Ip subnet and ethernet LAN behind your ADSL/cable modem
  - - - Data riser - vertically aligned switch/server rooms
      - flood wiring per floor - to faceplates from witch room
      - Ethernet switch stacks - one switch port per faceplate
    - - e.g. if an IP router is used to connect the whole building
      - Or may introduce one ethernet LAN per floor if there is an IP router on each floor
    - - Typically an ADSL router, with multiple Ethernet ports and 802.11 WiFi
      - Typically just one LAN, no internal routing
  - - - Lets you create physical loops, without breaking network
      - Algorithm decides a root bridge/switch (lowest MAC address)
      - Determines least cost paths to the root, forming a tree of paths
      - It then enables the tree-root ports on each switch
      - Other ports with a path may be enabled if topology changes (failure)
      - switches determine the root and the best paths to root
      - SPECIFICS NOT EXAMINABLE JUST BE AWARE OF IT
  - - - 12-bit value on 802.1Q tag
      - Can then place a switch port in specific VLAN(s)
      - Can carry multiple VLANs over one trunked uplink
- - - - Connectionless
        
        Sender just sends a datagram to a receiver
        
        No sequence numbers, no acknowledgements
      - If retransmission is required, the application has to do it - it is up to the application layer - It may or may not be necessary to resend
      - Some UDP applications may use a constant bit rate
      - Low overhead
        
        Because mo connection management being used
        
        Uses less bandwidth for the UDP header
    - - Allows sending of datagrams without establishing a connection
      - the UDP header is much simpler than TCP
      - Checksum is optional
      - Can multicast (one to many)
    - - Lossy/congested links can drop packets - higher protocols can send a request back to source if needed
      - Lower bandwidth links may drop packets as their buffers fill up - applications could detect this and tell server
  - - - Includes acknowledgements and retransmissions
      - Provides flow control/congestion control for segments it sends
      - TCP will adjust sending rate over time
      - Uses that capacity but tries also to avoid congestion
      - Receiver reassembles segments in the correct order
      - So TCP provides performance and reliability on an otherwise unreliable IP service
      - Connection Oriented
    - - Three way handshake (SYN,SYN-ACK,ACK) - Son opens connection (with random seq num), server ACKs, client ACKs. Connection is now established
      - Each side uses a sequence number
      - repeat packets can be discarded, lost ones resent
      - Common understanding of position in data stream
    - - ACKs are sent back by receiver
      - Sender must detect lost packets
      - By retransmission timeout - estimate when ACK is expected
    - - TCP uses a sliding window protocol to control the sending rate - receiver has limited incoming buffer size
      - Sender should not send data unless receiver indicates it has buffer space to accept it - otherwise will only have to resend later and wastes network bandwidth
      - The 'sliding window' is effectively the buffer space the receiver says it has available at any given time - this will change over time
      - The Sliding Window
        
        The sender sends a segment with a sequence number and starts a timer
        
        Receiver replies with an acknowledgement number showing next sequence number it expects to receive and its available window size
        
        If the sender's timer goes off before this is acknowledgement Is received the send retransmits
        
        If receiver says window size is 0, the sender may send a 1-byte probe to get a new window advertisement - or wait until receiver indicates it has capacity
    - - Maintained by the sender
      - The TCP congestion window indicates the number of bytes a sender may put into he network at any time
      - Packet loss is a signal of congestion
      - Runs alongside the reciever's sliding window
      - Use the smaller of the two windows when sending
      - The congestion window size starts low
      - Add one segment's worth per segment acknowledged before acknowledgement timer goes off
      - known as 'slow start'
      - If successful, this doubles the window every round trip
  - - - The solution used by YouTube
      - Commonly called web streaming
      - Client connects via TCP (http) and receives the video data, buffering it before playback
      - On a fast link, the client can typically buffer ahead if watching pre-recorded video
      - Problematic if link capacity not enough, and TCP backs off - you may see the 'hourglass of doom' - will stop playing rather than playing the 'broken' video
      - Not so well-suited for live video
- - - - To act as resolvers for internally sourced DNS queries, either for internal names or external names
      - To act as servers for external queries against internal names
  - - - These are essentially responsible for '.'
      - The root servers know where the DNS servers for each TLD are
  - - - It first asks a root DNS server - using the list of those pre-configured in the local resolver
      - If then iterates through the delegated DNS hierarchy
  - - - Root servers are required as the anchor for recursive DNS
      - In principle, there are only 13 root DNS servers
      - In practice, 350+ distributed around the world via anycast
        
        Anycast allows a client to reach the nearest instance server
        
        You can advertise the same IP, or small IP block, at multiple points on the internet
        
        Routers then learn of the nearest instance, topologically, via the routing system
        
        This means you will access a different instance, depending on where you are
      - Multiple instances of each root server
        
        A means to distribute a service
        
        Helps spread genuine load, and mitigate DoS attacks
- - - - Send directly to a destination on the same local subnet
      - OR forward to a router
    - - All hosts on a network have a routing table
      - May be built from information from DHCP or IPv6 RA
      - For most hosts, this is small and includes:
        
        Destination IP prefixes and the interface or next hop to use
        
        the local subnet that the host is connected to
        
        a catchall "default route"
      - Tells us how to route traffic
      - Routing table is a set of rules
      - There can be multiple routes for one destination
      - The most specific matching route is picked first - e.g. the route with the longest prefix
      - The metric determines the priority of routes with the same specificity - a lower metric means a higher priority
      - Prefix Aggregation
        
        In principle all Internet routers would need to know the presence of every subnet on the Internet
        
        In practice, this is not needed - a subnets prefix can be aggregated with other adjacent subnets
        
        An organisation might only advertise one route for its entire address space
        
        N.B IPv4 exhaustion makes prefix aggregation harder and routing tables larger
      - Beyond the default router
        
        We said that IP packets not delivered locally are sent via the default router
        
        The default router (and any router beyond it) needs to know where to send the packet next - it has its own routing table
        
        This could be very large and subject to frequent changes
        
        Manual configuration may not be feasible
        
        This is where routing protocols come in
  - - - Multihomed
      - Transit
      - Single-homed/stub
  - - - Interior gateway protocols are used within an Autonomous System - e.g. within a corporate network
        
        Distance Vector
        
        Talk only to directly neighbouring routers
        
        Exchange "best" route (shortest distance) information for any known prefixes with direct neighbours
        
        Routing Information Protocol (RIP)
        
        One of the oldest routing protocols (RFC 1058, 1988)
        
        Router sends its whole routing table periodically (every 30s) to directly connected routers
        
        Destination network (prefix) and distance (cost) in hops
        
        Receiving routers update their view of the best route (lowest distance) to a given network prefix
        
        limitations
        
        Updates only sent every 30s: "news" takes time to travel
        
        Updates are not acknowledged (UDP); message loss possibility
        
        Metrics are simple hop count values
        
        Limited to max value of 15 - a value of 16 means unreachable
        
        Thus only suitable for smaller homogeneous networks
        
        Routers don't have knowledge of network topology - can lead to 'count to infinity' problem - see Tanenbaum
        
        Authentication is MD5, which is broken
        
        Link state
        
        Talk to all routers to establish full knowledge of the routers/topology in a site
        
        Routers flood information messages describing their connected neighbours around entire site network
        
        Defacto enterprise routing algorithm usually IS-IS or OSPF
        
        Steps
        
        Discover neighbours & determine cost metric
        
        Flood message with this information to all routers
        
        Use recieved messages to build topology by computing shortest paths for prefixes served by any given ripper
        
        Messages are sent periodically, or any time a change in connectivity is detected
        
        Both ends of a link must agree for it to be valid
        
        All routers learn the full network topology
        
        Discovering neighbours
        
        Link cost is then determined: typically based on bandwidth and/or delay - could be based on other factors
        
        Neighbours are discovered with broadcast packets sent on all interfaces
        
        Building link state packets
        
        Each router creates link state packets based on neighbours and costs to reach them
        
        The packets are sent to all routers in the network: may be multicast or broadcast - sent periodically or when requested
        
        Computing shortest path tree
        
        Each route computes the best paths
        
        Uses Dijkstra's algorithm(from 1959)
        
        Determines shortest path through a graph from an initial node to any given destination
        
        Algorithm works by expanding from the starting node, considering cheapest neighbour with each iteration
        
        Thus the algorithm finds the next 'closest' node with each iteration, until it reaches the destination
        
        The tree is then used to populate the routing table
        
        Link state vs Distance vector
        
        Link state converges faster - e.g. OSPF can detect changes to topology and converge in seconds
        
        Link state is better at avoiding loops - Every node knows everything, can avoid loops - in distance vector, each node knows a little - relying on accuracy of information from prior nodes
      - Exterior gateway protocols are used between Autonomous Systems - e.g. the internet
        
        Inter-site routing uses exterior routing protocols
        
        Advertise your network prefixes to neighbouring networks
        
        You may or may not offer transit to other networks
        
        Policy is often more important than path costs
        
        De facto protocol is Border Gateway Protocol (BGP)
        
        Works between Autonomous Systems (AS)
        
        Distance-vector-like, but includes information about the AS path associated with a given route, cost of paths and other rich attributes
        
        BGP operation
        
        In configuration, specify IP of neighbours and AS
        
        Creates a BGP peering session - initially sends whole routing table, then incremental UPDATEs
        
        Then you advertise routes you know of to your neighbours - contains network prefix, prefix length, AS path, next hop ... - filtering can be applied to what is advertised and what is received
        
        Neighbour may then choose whether to use that route - can use knowledge of ASNs to decide whether to accept/ use the route
        
        You know the full path - so you can detect loops if your own AS is on a path you receive
        
        Downsides
        
        Relies on trust - a malicious peer can cause issues with your routing
        
        BGP is slow
        
        Updating takes a lot of effort
        
        Links can go up and down for a variety of reasons
        
        Update too quickly and you get flapping
        
        Default KEEPALIVE period is 60s, with a HOLDDOWN timer of 180s
        
        Routers have limited BGP routing table sizes
        
        Internet expansion and IPv4 exhaustion are pushing this limit on older hardware
        
        For many current routers, this is 1M IPv4 and 128k IPv6 routes
- - - - 3.4 x 10^38 IPv6 addresses
      - Current public pool: addresses start with 2xxx and 3xxx (4.2 x 10^37 addresses)
    - - Leading 0s in block can be omitted
      - A single set of repeated 0 blocks can be replaced by '::'
    - - Smallest subnet us a /64
      - Home/small business users should be given a /56
      - A site is commonly given a /48
      - Minimum IPv6 allocation from RIR is /32
      - /127 can be used for inter-router links
    - - 8 fields
      - fixed size (40-bytes)
      - Extension headers rather than options
        
        Sit between the IPv6 and transport layer headers
        
        Each extension header includes a "next header" field - can be daisy chained but need to be in a specific order
        
        A lot of extra functionality: Hop-by-hop options, destination options, routing, fragmentation, authentication, encapsulating security payload
        
        Each extension header is a multiple of 8-bytes
      - removes checksum
  - - - Router Solicitation: Host request for router information
      - Router Advertisement: Router information
      - Neighbour Solicitation: Equivalent to ARP "who has"
      - Neighbour Advertisement: Equivalent to ARP reply
      - Redirect: Router informing host of a "better" first-hop
    - - A host see or solicits a Router Advertisement (RA)
        
        The RA message carries the IPv6 network prefix (/64) to use
        
        The RA's source address implies the default router address
        
        Flags indicate how addresses can be included in a RA
      - Prefix information is sent by multicast
        
        Periodically (typically every 600s)
        
        On request (in response to a Router Solicitation)
    - - Hosts can auto configure basic network settings without a DHCPv6 server (RFC 4862)
      - A host using SLAAC builds its address from:
        
        A 64-but prefix determined from a Router Advertisement
        
        A 64-bit host part generated based on MAC address
      - Alternative method in RFC 7217:
        
        Does not embed MAC address
        
        Mitigates many of the security/privacy issues of RFC 4862
      - Privacy Extension
        
        Embedding the MAC address in the auto-configured global address causes a privacy issue
        
        Host part stays the same across networks
        
        Makes it easier to track users
        
        IPv6 Privacy Extensions (RFC 2941) added that use a random 64-bit host part
        
        Random IPv6 generated periodically (e.g. hourly) and used for outbound connections
        
        Host still has SLAAC-configured address for inbound connections
        
        Technical Benefits
        
        Removes the need for NAT
        
        Restores end-to-end connectivity
        
        Remove the need for NAT traversal
        
        More plug-and-play than IPv4: SLAAC works
        
        Streamlined header: more efficient routing and packet processing
        
        Fragmentation only occurs at sender:
        
        Hosts should use Path MTU Discovery (RFC 8201)
        
        Links must support an MTU of at least 1280
        
        Simpler for routers
  - - - IPv4 is on its last legs
      - IPv6 is mandatory
        
        Some ISPs ONLY give out IPv6
        
        Some sites/services are ONLY available on IPv6
      - Enables innovation/teaching/research
      - Supports new applications
      - Allows people to access your services
    - - Run both protocols on the same equipment - used in this uni
      - May need to rewrite/update existing applications
      - Need to choose when to use IPv4 or IPv6 - current "preference" is IPv6 addresses
      - Requires IPv4 address space going forward - can use NAT/CGNAT with global IPv6 addresses
      - Need to secure (e.g. firewall) both protocols
      - DS-Lite
        
        Native IPv6 , tunneled, NATed IPv4
        
        encapsulation can run on client device
        
        EE use it for their IPv6
        
        Makes hosting of services on IPv4 difficult
        
        Can be extended with global IPv4
    - - 6in4 encapsulates IPv6 packets in IPv4 packets
      - Teredo: encapsulated IPv6 in UDP IPv4: uses a well-known IPv4 host on the internet to relay IPv6 traffic, work with no co-operation of local network - bypasses security
      - Cana lso use VPN to distribute IPv6 over IPv4
    - - some methods suited to certain scenarios - no best solution for all scenarios
      - IPv4 hosts will be around for a long-time, with transition ongoing for many years
      - Initial focus is on dual-stack deployment
      - But use of IPv6-only is growing, esp. with mobile operators
    - - Many major sites have IPv6
      - Cloudfare free domains are IPv6
      - Several major ISPs support IPv6
      - Many in early stages of roll-out
      - Some stubbornly ignore IPv6
      - 2021 - 35% of global google users use IVp6
    - - Time and money
      - Hardware support - many older home routers lack IPv6 support
      - Regular users don't want to learn something new
      - Chicken and egg situation with ISPs and content provider
      - "Peering wars" between Tier-1 network providers
  - - - IPv6 hosts can essentially pick their own addresses - may use many privacy addresses over time
      - To track which devices are used where, we can poll switches and routers for MAC table and ARP table information: build a database of mappings, switch port - device MAC, MAC address - IPv4/IPv6 address
      - In ECS, an open source network monitoring package called NAV used to be used for this
- - - - Common email protocol
      - Designed in 1986
      - Server listens on port 143 (993 for secure)
      - Client keeps a TCP connection open - to send requests or receive notifications
  - - - First major revision, published in 2015
      - Header compression - now binary using HPACK
      - Server push
      - Multiplexing multiple requests over a single TCP connection
    - - this month it is an "expired" Internet draft (but not abandoned)
      - but runs over QUIC and is widely in use (which helps previous TCP issues)
  - - - From Microsoft world
      - TCP port 445 or UDP ports
      - provides authentication, file locking
      - Samba in Linux provides it
    - - used mainly in Unix/Linux
      - Common between servers
      - Has to cope with very large files and file numbers
- - - - Stateless firewalls are limited
      - Modern firewalls are stateful
        
        Inspect TCP/IP header fields and keep track of connections
        
        Have rules for related and established
      - A common rule set for home firewall is:
        
        Deny all inbound unless related/established
        
        Allow all outbound
    - - Even more advanced firewalls can actuate the application layer (layer 7 firewalls or Application Layer Gateways)
      - Inspect the contents of the packet to make sure that that packet that looks like HTTP on port 80 is actually HTTP
      - Doesn't necessarily work with encrypted traffic
  - - - Monitora a network looking for malicious activity or policy violations
      - May be signature based or use anomaly-base detection
    - - Make is so that if an unauthorised device plugged in, it gets nothing
      - Simplest "solution" is MAC address filtering - easy to circumvent and an administrative overhead
      - A better solution is port-based network access (802.1x)
      - 802.1x
        
        Uses the Extensible Authentication Protocol (EAP) to authenticate clients
        
        Blocks the switch port/wireless client until they authenticate successfully
        
        Involves 3 parties
        
        A supplicant (client)
        
        An authenticator (switch or access point)
        
        An authentication server (e.g. LDAP using RADIUS)
  - - - Site-to-site VPN
        
        Securely connects two networks
        
        e.g. a branch office to head office
      - Remote-access VPN
        
        Individual clients connect to a VPN server
        
        e.g. Globalprotect
  - - - Wireless connection is inherently less secure than a wired connection
        
        signal is not constrained by wires
        
        Anyone within range can listen in or participate
      - Simple networks behave like hubs as every client recieves all traffic
    - - Most contemporary networks use some encryption to restrict access and ensure data integrity
      - Open "hotspots: are used to provide complimentary/ public access
    - - the original method of encryption for 802.11 Wireless Networks
      - Intended to provide data confidentiality comparable to a wired network
      - 40-bit or 104-bit key and 24-bit IV
      - Uses the RC4 stream cypher
      - Proved to be insecure in the early 2000s
    - - WPA: 2003
        
        an intermediate solution to the weakness of WEP
        
        Uses Temporal Key Integrity Protocol TKIP
      - WPA2: 2004
        
        more secure and more complex than WPA
        
        Uses AES as the encryption algorithm
      - WPA3: 2018
        
        Same encryption "strength" as WPA2
        
        Improves security of initial key exchange
      - WPA-Personal: uses pre-shared keys
      - WPA-Enterprise: uses 802.1x authentication
      - WPA PSK Vulnerabilities
        
        Until recently, attacks focused on the human element or other weak/predictable systems
        
        weak auto-generation of ISP-set default keys
    - - Intended to make it easier to connect to a WPA-protected network
      - User enters an 8-digit pin or presses a button on the access point for initial connection
      - BUT the pin can be brute-forced quickly
      - Prevailing recommendation is to disable WPS
- - - - get, put, post, delete, etc
      - copies codes from HTTP (e.g. 404)
      - Makes interoperability with The Web simple
    - - low memory footprint
      - basic packets should fit in an 802.15.4 radio frame
    - - A more modern lightweight design than MQTT
      - Prefers UDP
      - Binary format for protocol
      - DTLA security option
      - Block transfers
      - Resource discovery
      - push notifications
      - cache model
      - Multicast support
    - - is very small
      - Version (01)
      - Type (eg 00 confirmable)
      - Tkl: Token length
      - Code is method (1-10)
      - ID helps match message
      - Token is used to match response
      - Option can mean payload is a URL string
      - Payload can be anything
- - - - use it for a DNS amplification attack
      - Spoof source IP address to send lots of (small) DNS requests
      - DNS is UDP so connectionless
      - DNS server replies to spoofed address with lots of (large) DNS responses
      - Destination is inundated with traffic and falls over
      - PROFIT
    - - DNS is not signed or encrypted
      - Can be taken further to DNS cache poisoning:
        
        Attacker supplies a bogus response to a caching resolver
        
        Only needs to know the queried server's IP address and guess (or snoop) a 16-bit transaction ID
        
        All users of the resolver now use the bogus entry
      - Or just malicious network operator: Include dodgy entries in their DNS server
    - - Your connection to a server may be encrypted - what you are doing is obscured
      - But DNS is not encrypted:
        
        anyone sniffing the connection can see your DNS requests
        
        Where you are doing it is not
      - What can a 3rd party learn from DNS queries and their timing?
    - - DNSSEC addresses DNS insecurity
      - Developed > 10 years ago, still being deployed - allows responses to be signed BUT responses are larger AND complexity managing signatures
    - - Mitigating DNS amplification
        
        Blocking the spoofed IP doesn't help: It blocks the victim, not the attacker
        
        Block resolver lookups from non-local IP addresses: e.g. restrict lookups to those from your organisation or customers
        
        Ignore certain request types (<root> and ANY)
        
        Protect against IP spoofing on your network with BCP 38
      - Mitigating DNS amplification requires the DNS server administrator to configure things properly
      - Open resolvers are bad - request types that give large responses
      - The other half of the DNS amplification problem is the ability of attackers to spoof source IP addresses Hence BCP38 - one of the best DDoS defences - deployment of BCP38 at 55%
    - - A protocol used by domain registrars and registries to manage domain names
      - Used by most gTLDs and ccTLDs
      - XML-based
  - - - Uses more systems to deprive the victim of the targeted resource (usually bandwidth)
      - Is harder to stop as there are multiple sources and many of them may be expected sources in normal conditions
    - - Compromised computers/routers/loT devices in a botnet
      - Volunteers answering a "call to action"
    - - Exploits the TCP protocol
        
        Attack the connection handshake
        
        Attacker send a TCP SYN packet
        
        Server replies with SYN ACK
        
        Attacker never send final ACK- so victim left waiting
        
        Consumes state on server
        
        Many half-open connections
        
        eventually cannot respond to legitimate requests
        
        Doesn't require much bandwidth
    - - A fairly crude UPD/TCP attack tool
      - Branded as a "stress tester"
      - Used to form voluntary botnets
      - Can be (partially) mitigated by blocking UDP & ICMP
    - - Replacement for LOIC
      - Uses HTTP Flood: e.g. lots of POSTS stress the server
      - Works with far fewer users than LOIC
    - - Uses a single machine (or mobile phone)
      - Opens lots of connections
      - Exact implementations depends on web server
      - Can be mitigated with (reverse) proxies, limiting the number of connections per-IP or increasing capacity
    - - Sends a large number of IPv6 RA multicasts
      - Consumes CPU on victim machines
      - "easy" to exploit
      - Windows 10 now stops joining new networks after 100
    - - Anycast/content delivery networks: spreads the load across lots of networks/systems
      - Responsive network team: block attacks as they occur, redistribute services across different connections
      - Good relationships with upstream providers: get them to help blocking attack traffic
      - Make sure your systems are patched and configures securely
      - DDoS is easy to attack with, hard to defend against
- - - - Data and computation are distributed over networked machines
    - - Reads/writes from/ to memory/disk do not require passing messages over the network (internal buses are used)
    - - Concurrency
      - no global clock
      - independent failure
    - - Concurrency is a design choice - threads/processes can communicate through shared memory or message passing
      - global clocks
      - Simpler failure model
    - - To share resources
        
        e.g. client-server paradigm
        
        Concept of service, to bridge the gap between service provider and service consumer
      - To have redundancy
        
        Better fault tolerance
        
        Consistency problem
      - Opportunities to parallelise computation
  - - - Move core services to remote servers
      - Interact to store/retieve data and use services through a web browser/app
    - - Volume: quantity of data
      - Variety: diversity of types of data
      - Velocity: speed data is generated and needs to be processed
      - Veracity: quality and value of data
      - New requirements for distributed systems
    - - Centralisation can lead to
        
        Performance bottleneck
        
        Single-point-of-failure
        
        Censorship
      - Distributed systems as a way to remove centre-based architectures
        
        peer-to-peer network
        
        blockchain
    - - Pervasive network of sensors and actuators
        
        smart home
        
        Smart building
        
        Smart vehicles
        
        Industry 4.0
      - IoT devices are not general purpose and have limited resources
        
        Need to (aggregate and) send data to more powerful devices for analysis
        
        Usually, data sent to the cloud
    - - More powerful devices to be deployed at the edge of the network
      - Edge devices can carry out (part of) the computation where data is generated
    - - Even more resourceful devices can be placed between the edge and the cloud
      - Fog devices can carry out (part of) the computation very close to where data is generated
  - - - Variety and difference of networks, hardware, operating systems, programming languages, implementations
      - Middleware: software layer that provides a programming abstraction as well as masking the heterogeneity of the underlying networks, HW, OSs and programming languages
    - - An open system can be extended and reimplemented in various ways: to add more computational/storage resources, To integrate additional services
      - The more easily a system can be extended, the more open it is
      - It requires key interfaces to be published: similar to standardisation but less formal and (thus) faster
    - - A sytem is scalable if it remains effective when there is a significant increase in: the number of resources and/or the number of users
      - Controlling the cost of physical resources - ease to add resources to meet growing demand
      - Controlling the performance loss - algorithms performance should remain acceptable as problem size grows
      - Preventing software resources running out - set reserve the proper size for core data structure
      - Avoiding performance bottlenecks - avoid centralisation
    - - Detect failures
      - Mask failures
      - Tolerate failure
      - Recovery form failure
    - - More clients can try to access the same resource at the same time
      - Solution 1: Serialise access : safe but limits throughput
      - Solution 2: allow fro concurrent accesses: maximise throughput, can lead to inconsistencies
      - Need to encapsulate the resource and regulate accesses based on application specific requirements
    - - Concealment from the user and the application programmer of the separation of components in a distributed system, so that the system is perceived as a whole rather than as a collection of independent components
    - - e.g. ability to meet deadlines
- - - - Use models to represent distributed systems and capture their relevant aspects
      - Develop general solutions for classes of distributed systems
      - Reuse those solutions for classes of distributed systems
    - - Computers
      - Interconnecting networks
    - - Architectural elements (computational and communication tasks)
      - Architectural patterns (how to combine elements)
    - - Interactions models
      - Failure models
      - Security models
  - - - components located at computers
      - Computers interconnected by a network
      - components communicate by passing messages over the network
    - - early distributed systems
      - internet-scale distributed systems
      - Contemporary distributed systems
      - Distributed systems of systems
      - more info on each in lecture
  - - - Types of components involved
      - Their inter-relationships
      - How they interact among each other
    - - Availability
      - Performance
      - Adaptability
      - Security
    - - How to identify the building blocks of a distributed system?
        
        communicating entities what are the entities that are communicating?
        
        communication paradigms how do they communicate?
        
        roles and responsibilities what roles/responsibilities do they have?
        
        placement how are they mapped on to the physical distributed infrastructure?
      - Communicating entities
        
        System-orientated perspective (lower level, i.e. what is running?)
        
        Processes communicating via inter-process channels
        
        If no process abstraction is available (e.g. sensors), nodes
        
        Threads (if available) as lightweight execution means
        
        Problem orientated perspective (higher level, i.e. can I abstract what is running?
        
        Objects (provided interface)
        
        Components (both provided and required interfaces)
        
        Web services (web-based, though to be accessed worldwide over the internet)
      - Communication Paradigms
        
        Inter-process communication (network level)
        
        message passing
        
        sockets
        
        Remote invocation (communication protocols based on a two-way exchange
        
        Request-reply protocols
        
        remote procedure call (RPC) - request the execution of a remote function and receive the response
        
        remote method invocation (RMI)
        
        Match with problem-orientated communicating entities
        
        Object-orientated RPC, i.e. invocation of a method of a remote object
        
        Limitations of inter-process communication & remote invocation
        
        space coupling: senders need to know how to reach the receivers
        
        time coupling: senders and receivers need to be up and running at the same time
        
        Indirect communication: time and space decoupling between senders and receivers
        
        Group communication
        
        publish-subscribe systems
        
        message queues
        
        tuple space
        
        distributed share memory
      - Roles and Responsibilities
        
        Client-sever
        
        clients send requests to servers and receive responses back
        
        Clients enabled to access shared resources managed by servers
        
        servers can be clients of other servers
        
        peer-to-peer
        
        All involved entities play similar roles
        
        clients themselves can become part of the p2p network and provide their resources to support the p2p service
        
        enables decentralised approaches
      - Placement
        
        How to map identified elements on to the underlying physical infrastructure
        
        placement impacts on
        
        performance
        
        reliablilty
        
        security
        
        Aspects to take into account for the design
        
        communication patters
        
        Reliability of machines
        
        Application workload
        
        Quality of communication channels
        
        mapping of services to multiple servers
        
        replicate application logic
        
        data object can be either
        
        partitioned over the servers
        
        replicated over the servers
        
        replication
        
        pros: fault tolerance, load balancing
        
        cons: need to ensure data consistency
        
        partitioning
        
        pros: no overhead for data consistency
        
        cons: no real fault tolerance
        
        Caching
        
        a cache is a store of recently used data objects that is closer to the client(s)
        
        Caches can also be co-located with clients
        
        web proxy servers can also be used to access web servers through a firewall
    - - key patterns
        
        layering - logical organisation
        
        tiering - physical organisation
        
        thin clients
        
        proxying
        
        brokerage
- - - - behaviour and state are determined by the intertwining of the sequential steps of each involved process
      - A distributed algorithm includes the transmission of messages
      - processes proceed at different rates and sen/receive messages at different rates
      - the system state is distributed as well, no global notion of system state - global system state: combination of the local state of all involved processes
    - - Latency: from start of transmission to start of reception, measured in seconds
      - Bandwidth: total amount of information that can be transmitted over it in a given time
      - Jitter: variation in the time it takes to deliver a series of messages
    - - If two processes running on different physical machines read their clocks at the same time, they would get different time values
      - Corrections can be applied through synchronisation protocols, but the accuracy is affected by network delays
      - no global notion of time, cannot reply on local clocks for synchronisation purposes
      - Synchronous vs Asynchronous Systems
        
        Difficult to set limits
        
        process execution time
        
        message latency
        
        clock drift rate
        
        estimations are possible but relying on them is not safe
    - - Assumptions: known upper and lower bounds on processing time, network latency and clock drift rate
      - simpler design
      - useful when starting the design of a new distributed algorithm, to draft a preliminary version and validate it
    - - assumptions: no bound on process execution speed, message transmission time and clock drift rate
      - Harder to design
      - models the internet and real systems much better than synchronous assumptions
      - Any solution that is valid for an asynchronous distributed system is also valid for a synchronous one
  - - - Channel Omission failures
        
        send omission failure
        
        receive omission failure
      - Arbitrary (Byzantine) failures
        
        Send/transmit arbitrary messages at arbitrary times
        
        commit omissions
        
        a process may stop or take an incorrect step
        
        A malicious process
      - timing failures
        
        relevant in synchronous systems
        
        time limit on process execution time, message delivery time or clock drift rate are violated
      - Process Omission failures
        
        crash: process has halted and will not execute any further steps of its program ever
        
        Can other processes detect that a process has crashed?
        
        Synchronous model
        
        Can use timeout: if a process P does not reply to a message within a certain timeout, P is considered as crashed
        
        All the processes can detect a crashed process: fail-stop failure
        
        Asynchronous model
        
        Maybe the process is not replying because it is slow of the message is not arrived yet
        
        Other process cannot reliably detect a crashed process: crash failure
    - - Masking
      - conversion
    - - VALIDITY: any message in the outgoing message buffer is eventually delivered to the incoming message buffer
      - INTERGRITY: the message received is identical to one sent, and no messages are delivered twice
  - - - threats to processes: identity spoofing, malicious code
      - threats to communication channels: copy/alter/inject messages
      - Denial of Services
- - - - for resource sahring, e.g. concurrent updates of database records, files, etc
      - to agree on actions/values e.g. agree on sensor readings, agree to proceed
      - to dynamically assign the role of co-ordinator
  - - - Common methods of communication: shared memory, sending point-to-point or broadcast messages, executing remote procedure calls, etc
    - - Synchronous, asynchronous, partially synchronous
    - - Typical domain problems inc. problems or resource allocation, communication, consensus among distributed processors, database concurrency control, deadlock detection, global snapshots, etc
- - - - The happened-before relation (denoted as →) on a set of events satisfies the ffg. conditions:
        
        If a and b are events in the same process, & a occurs before b, then a → b
        
        If a is the event of sending a message by one process & b is the event of receipt of the same message by another process, then a→b
        
        if a→b & b→c, then a →c, i.e. happened-before is a transitive relation
        
        → is a partial order (reflexive, transitive, antisymmetric) [assume a → a holds]
        
        a.b are concurrent (written a||b) if neither a →b nor b→a i.e. two events a and b are concurrent iff no path exists either form a to b or form b to a
    - - To determine whether an event a happened before an event b, either a common clock or a set of perfectly synchronised clocks will be needed
      - In a distributed system, no perfect clock synchronisation, impossible to order events in different processes by time of occurrence
      - but we can use event causality to obtain a partial ordering of events
        
        events within same process ordered uniquely by local clock time
        
        certain pairs of events in different processes (namely send and receive) can also be ordered
      - Lamport's Logical clocks
        
        To determine if a happened before b, you'll need a common clock or perfectly synchronised clocks
        
        Globally synchronised clocks are difficult to achieve in Distributed Sytems
        
        Lamports Logical clock concept is a way to define the happened-before relation in a distributed system
        
        It associates a timestamp (may just be a monotonically increasing number) with each system event
        
        Here, each process Pi has a logical clock Ci associated with it assigning a number Ci(a) to an event a in Pi
        
        Conditions to satisfy
        
        If a and b are 2 events within the same process Pi, & a occurs before b, then Ci(a) < Ci(b)
        
        If Pi send a message a and is received as b by Pj, then Ci(a) <Cj(b)
        
        A clock Ci associated with a process Pi must always go forward, i.e. Ci is monotonically increasing
        
        Lamport's algorithm
        
        Pi increments Ci before assigning a timestamp to an event
        
        When Pi sends a message m at event a, m contains a timestamp Tm = Ci(a)
        
        When Pj receives m, it sets Cj greater than or equal to its present value, but greater than Tm
      - properties
        
        If C(a) I the timestamp for an event a at whatever process it occurred
        
        Then: a→b implies C(a) ⩽C(b)
        
        The opposite is not true, i.e. C(a) is not necessarily greater than C(b)
    - - Logical clocks (C) of 2 events cannot reveal if they are casually ordered, i.e.
        
        in C, a→b → C(a)⩽C(b)
        
        we are not sure if when C(a)⩽C(b), it really implies a →b
      - Goal of Vector Clocks, V, is to detect causality
      - Vector clocks, V define
        
        A mapping V: {Ev} → [int]// from set of events to integer
        
        an order <: ∀a, b : Ev . a< b ↔ V(a) < V(b)
      - each process, p keeps its own vector clock, Vp, & uses it to timestamp local events
      - Timestamps are attached to messaged and updated upon receipt of messages
    - - Ecyernal clock synchronisation: synchronisation of the computer clock with real-time clocks. An often used external time source is the Coordinated Universal Time (UTC)
      - Mutual (or internal) synchronisation of the clocks of the different system nodes. Particularly useful if an application requires a consistent view of time across all nodes of the distributed sytem
      - Two broad classifications
        
        Centralised Algorithms: one node, called the time server node, has real-time receiver. The clock time of this node is regarded as correct and used as the reference time
        
        Passive Time Server Centralised Algorithm, e.g. Critian's algorithm
        
        Active Time Server Centralised Algorithm, e.g. Berkeley algorithm
        
        Drawbacks
        
        May be unreliable due to single-point-failure
        
        From a scalability point of view it is generally not acceptable to get all the time requests serviced by a single time server
        
        Distributed Algorithms: each nodes clock is independently synchronised with real time
- - - - Assume reliable communication channels between processes - all messages eventually delivered
        
        reliable communication protocol masks communication failures
        
        Additional bounds on delivery time in synchronous systems
      - failed links between processes eventually repaired / circumvented
    - - processes only fail by crashing (no arbitrary failures)
        
        correct process: does not fail at any point in the execution under consideration
      - can build failure detectors
        
        service that processes queries regarding process failure - usually implemented using local failure detectors, one for each process
        
        unreliable failure detectors
        
        return hints: unsuspected / suspected
        
        most practical systems implement these
        
        reliable failure detectors
        
        return Unsuspected hint/ failure guarantee
        
        difficult to implement in practice
    - - omission failures: process/channel fails to perform intended actions
      - arbitrary (byzantine) failure: wrong process/channel behaviour
      - Timing failures (in synchronous systems)
  - - - prevent interference / ensure consistency when accessing shared resources
      - e.g. avoid concurrent updates, implement atomic operations, medium access control in Ethernet/wireless networks
    - - minimise number of messages sent in each entry /exit operation
      - minimise client delay when entering /exiting CS
      - minimise synchronous delay (time between one process exiting and another entering critical section
    - - (N processes)
      - network is reliable but asynchronous
      - processes do not fail
      - processes spend finite time in critical section
    - - safety: at most one process executes in critical section at any time
      - liveness: requests to enter/exit critical section eventually succeed
      - fairness: accesses to critical section in happened-before ordering of requests
- - - - central server for mutual exclusion
      - sequencer for totally ordered multicast
      - coordinator for distributed transaction
    - - at beginning
      - when leader fails - how to detect/ recover from this?
- - - - A multicast by a group member is atomic, when the message is received either by every functioning group member, or by no member at all. Atomic multicast is a basic requirement of all group-orientated activities
      - GUARANTEES
        
        a correct process will eventually deliver the message, as long as the multicaster does not crash
        
        only group members receive messages ent to group
      - uses primitives B-multicast and B-deliver
      - implementation uses reliable one-to-one send operation
    - - tolerates process crashes
      - assumes closed group
      - delivery guarantees
        
        integrity
        
        validity
        
        agreement
      - algorithm is inefficient
      - correctness
        
        integrity: yes
        
        validity: yes
        
        agreement: yes
    - - possible guarantees
        
        FIFO orderif a correct process issues multicast(gym) and then multicast(gym') then any correct process delivers m before m'
        
        causal order: if multicast(gym) -> multicast(gym') then any correct process delivers m before m'
        
        total order: if a correct process delivers m before m', then any correct process delivers m before m'
      - casual order implies FIFO order
      - total order does NOT imply FIFO/casual order
      - above do not assume reliable multicast
    - - multicast operation more appropriate for comm. to a group of processes
        
        more efficient
        
        group membership transparent to sender
      - IP multicast: packets are sent to multicast groups
        
        dynamic group membership
        
        can send even if not a member
        
        unreliable
        
        omission failures
        
        does not tolerate process (multicast router) failure
        
        no guarantees on message or order
      - types
        
        closed when size remains unchanged
        
        open where membership changes form time to time
        
        peer-to-peer where all members are equal, and each member communicates with its peers to sustain the group's activities
        
        hierarchical where one member is distinguished from the rest, and comms usually go through this person
  - - - group membership is known
      - reliable one-to-one comm
      - processes only fail by crashing
    - - multicast(g,m) sends message m to all members of group g
      - deliver(m) delivers the message m to the calling process (different from receiving the message)
- - - - crash failures
      - Byzantine failures
  - - - processes begin in undecided state
      - each proposes initials value drawn from set D
    - - exchange values
      - each process decides on a value and enters decided state
    - - correct processes must reach consensus even in the presence of crashes, or even arbitrary failures
  - - - porcesses relibaly multicast proposed value to others
      - wait until all N values collected
      - decide through majority function
    - - termination and agreement guaranteed by use of reliable multicast
      - majority vote ensures integrity
    - - crash failure: process stops sending values after a while; difficult to detect in asynchronous system
      - arbitrary failure, either due to bug or deliberate; difficult to detect even in synchronous system
  - - - collect all processes in group
      - each process Pi RTO-multicasts its value vi to g
      - each process Pi chooses di as the first value it RTO-delivers
- - - - used CP replication approaches
      - require global ordering of updates
      - when updates are committed, replicas need to reach agreement on global ordering
      - different models, depending on guarantees on total ordering
    - - used by AP replication approaches
      - no global ordering, more relaxed guarantees on consistency
- - - - Acknowledgements are redundant, since requests are followed by replies
      - connection establishment overhead
      - flow control redundant
    - - requestId
      - Identifier of sender process
    - - Process crashes
      - channel omission failures
      - Messages not guaranteed to be delivered in the same order they are sent
    - - Return an error code/ exception
      - Retransmit
    - - If duplicate request received after the reply has been sent, the server needs to serve the request again
      - Idempotent operations (e.g. get_value vs update_value)
    - - To avoid re-execution
      - Memory cost: rely on client's next request, replies discarded after some time
  - - - The encoding and decoding of parameters and results
      - The passing of messages
      - The preserving of the required semantics for the procedure call
    - - Programming with interfaces
      - RPC call semantics
        
        Maybe semantics: no fault-tolerance measures applied
        
        Omission failures
        
        Crash failures
        
        At-least-once semantics: retransmission of request messages
        
        Crash failures
        
        Arbitrary failures
        
        At-most-once semantics: retransmission of request messages, duplicate filtering, retransmit reply
        
        Crash failures
      - Transparency
  - - - Use interfaces
      - Built on top of request-reply protocols
      - Transparency
    - - Can use the full expressive power of object-orientated programming
      - All objects have unique object references and can be passed as parameters
    - - Object references
      - actions upon method invocation
      - Exceptions
      - garbage collection
    - - Remote object references
      - Remote interfaces
      - Actions upon method invocation
      - Garbage collection
      - Exceptions
- - - - Focusiong on interfaces rather than implementation
      - Designing simple and generic interfaces
      - Opting for asynchronous rather than synchronous communication
  - - - Uses a simple minimum uniform set of operations- HTTP methods: GET, PUT, POST, DELETE
      - Focuses on resources rather than the operations allowed on them - each resource can be accessed through a specific, unique URL
      - Allows for looser coupling compared to SOAP because it removes the dependency on application-specific APIs - the emphasis on the manipulation of data resources rather than on interfaces
      - Stateless
      - Directory structure-like URIs
      - XML/JSON data format
- - - - The lost update problem
      - Inconsistent retrials problem
      - Conflicting operations and serial equivalence
        
        A pair of operations conflicts if their combined effect depends on the order in which they are executed
        
        For two transactions to be serially equivalent, it is necessary and sufficient that all pairs of conflicting operations of the two transactions be executed in the same order at all of the objects they both access
      - Locking, optimistic concurrency control, timestamp ordering
    - - Dirty reads Recoverability of transactions and Cascading aborts : if T aborts, any other. transactions that have seen the effects due to U must be aborted too
      - Premature writes
      - Strict executions of transactions:
        
        To avoid dirty reads and cascading aborts, transactions are only allowed to read objects that were written by committed transactions delay read operations
        
        To avoid premature writes, transactions are only allowed to write objects that were written by committed transactions delay write operations
  - - - Allows servers to abort unilaterally
      - Voting phase and completion phase
      - Failure handling based on timeout