[{"content":"Every fresh VPS (Hetzner, DigitalOcean, your homelab Proxmox box) boots up with the same problem: a root login over password, no firewall, and nothing installed. Before you deploy anything, spend 15 minutes on this checklist.\nOrder matters. Firewall rules and SSH hardening can lock you out permanently if done in the wrong sequence. Do the steps in this order and you won\u0026rsquo;t have to rescue-boot your way back in.\n1. Update everything first Connect as root and patch the box before touching anything else.\napt update \u0026amp;\u0026amp; apt upgrade -y A fresh image is usually weeks behind. Running services against a known-CVE kernel is the easiest way to get owned in the first hour.\n2. Create a non-root user with sudo Running everything as root is how one typo becomes a disaster. Create your day-to-day user now:\nadduser deploy usermod -aG sudo deploy adduser prompts for a password. Pick a strong one — this password doubles as your sudo password until SSH keys are set up (next step).\nVerify the user is in the sudo group:\nid deploy # uid=1000(deploy) gid=1000(deploy) groups=1000(deploy),27(sudo) 3. Add your SSH key (from your laptop) Do this before touching any SSH setting. Generate a key locally if you don\u0026rsquo;t have one:\n# on your laptop, not the server ssh-keygen -t ed25519 -C \u0026#34;laptop\u0026#34; ssh-copy-id deploy@\u0026lt;server-ip\u0026gt; ssh-copy-id appends your public key to ~/.ssh/authorized_keys on the server. Test it in a new terminal before going further:\nssh deploy@\u0026lt;server-ip\u0026gt; # should log in without asking for a password Keep this first SSH session open for the rest of the guide. If something goes wrong, it stays connected.\n4. Harden SSH: disable passwords, enable verbose logging Now that key login works, kill password auth. Password logins are what bots brute-force all day. Edit /etc/ssh/sshd_config (or drop a file in /etc/ssh/sshd_config.d/):\nsudo nano /etc/ssh/sshd_config Set these lines (uncomment if needed):\nPasswordAuthentication no ChallengeResponseAuthentication no PermitRootLogin prohibit-password LogLevel VERBOSE PasswordAuthentication no — keys only from here on. ChallengeResponseAuthentication no — closes the keyboard-interactive backdoor, which would otherwise still prompt for passwords. PermitRootLogin prohibit-password — root can only log in with a key. LogLevel VERBOSE — logs the key fingerprint on every login attempt. This is your SSH audit trail. Validate the config before restarting — a typo here breaks all SSH logins:\nsudo sshd -t \u0026amp;\u0026amp; sudo systemctl restart sshd sshd -t exits silently on success and prints errors on failure. Only restart when it passes.\nFrom a second terminal, test a fresh login before closing your first session. If the second session works, you\u0026rsquo;re safe.\nCheck recent logins. Verbose logging records a key fingerprint for each attempt:\njournalctl -u ssh --since today | grep -E \u0026#34;Accepted|Failed\u0026#34; Accepted publickey for deploy lines confirm key logins; Failed password lines show brute-force attempts bouncing off your disabled password auth.\n5. Firewall: deny everything, allow only SSH UFW is Ubuntu/Debian\u0026rsquo;s simple frontend to iptables. Default-deny all inbound traffic, then punch a hole only for SSH:\nsudo apt install -y ufw sudo ufw default deny incoming sudo ufw default allow outgoing sudo ufw allow ssh sudo ufw enable Answer y when it warns you about disrupting SSH — allow ssh was added before enabling, so your session survives. This is why step 5 comes after SSH is working.\nVerify the ruleset:\nsudo ufw status verbose Expected output:\nStatus: active Logging: on (low) Default: deny (incoming), allow (outgoing), deny (routed) New profiles: skip To Action From -- ------ ---- 22/tcp ALLOW IN Anywhere 22/tcp (v6) ALLOW IN Anywhere (v6) If you later run a service (say, Caddy on 443), open it the same way: sudo ufw allow 443/tcp. When in doubt, sudo ufw status numbered and sudo ufw delete \u0026lt;number\u0026gt; to remove a rule.\n6. Install the essentials Small, useful, nothing dev-stack-yet:\nsudo apt install -y git fzf curl htop fail2ban git — you\u0026rsquo;ll need it for cloning and config next. fzf — the fuzzy finder. Ctrl+R history search and Ctrl+T file completion become instant-fuzzy once it\u0026rsquo;s installed. Load the keybindings with source /usr/share/doc/fzf/examples/key-bindings.bash or add it to ~/.bashrc. curl — every install script and health check starts with curl. htop — top for humans; check load and memory at a glance. fail2ban — watches the SSH logs from step 4 and bans IPs after repeated failed attempts. Works out of the box on Debian/Ubuntu with the sshd jail enabled by default: sudo systemctl enable --now fail2ban sudo fail2ban-client status sshd 7. Configure git Even on a server, git needs an identity before its first commit:\ngit config --global user.name \u0026#34;Ramesh Kumar\u0026#34; git config --global user.email \u0026#34;ramesh@example.com\u0026#34; git config --global init.defaultBranch main git config --global core.editor nano git config --global credential.helper \u0026#34;cache --timeout=3600\u0026#34; Verify the configuration:\ngit config --global --list 8. Turn on automatic security updates (optional but free) Ubuntu and Debian can patch themselves for security CVEs without you:\nsudo apt install -y unattended-upgrades sudo dpkg-reconfigure -plow unattended-upgrades Answer Yes. It installs security updates daily and emails root on failures. Kernel updates still need a reboot — cat /var/run/reboot-required tells you when.\n9. Install Docker and add your user to the docker group Containers are the next thing most servers end up running. Install Docker from Docker\u0026rsquo;s official repo — the distro packages lag badly:\nsudo apt install -y ca-certificates curl sudo install -m 0755 -d /etc/apt/keyrings sudo curl -fsSL https://download.docker.com/linux/debian/gpg \\ -o /etc/apt/keyrings/docker.asc sudo chmod a+r /etc/apt/keyrings/docker.asc echo \u0026#34;deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] \\ https://download.docker.com/linux/debian $(. /etc/os-release \u0026amp;\u0026amp; echo $VERSION_CODENAME) stable\u0026#34; \\ | sudo tee /etc/apt/sources.list.d/docker.list \u0026gt; /dev/null sudo apt update sudo apt install -y docker-ce docker-ce-cli containerd.io \\ docker-buildx-plugin docker-compose-plugin On Ubuntu, swap debian for ubuntu in the two download URLs.\nAdd your user to the docker group so every command doesn\u0026rsquo;t need sudo:\nsudo usermod -aG docker deploy Group membership only applies to new logins — log out and back in (or run newgrp docker in the current shell), then verify:\ndocker run --rm hello-world docker compose version One gotcha to know: Docker publishes container ports directly in iptables, bypassing UFW. A -p 8080:80 container is reachable from the internet even with UFW\u0026rsquo;s deny-incoming default. Keep that in mind before exposing anything.\nThe final checklist Run through this before you call the server done:\n# 1. key login works, password login refused ssh -o PreferredAuthentications=password deploy@\u0026lt;server-ip\u0026gt; # -\u0026gt; Permission denied (publickey) ... good # 2. firewall active with ssh open sudo ufw status | grep -q \u0026#34;22/tcp.*ALLOW\u0026#34; \u0026amp;\u0026amp; echo \u0026#34;ufw ok\u0026#34; # 3. sshd config valid, password auth off sudo sshd -T | grep -Ei \u0026#34;passwordauthentication|permitrootlogin\u0026#34; # 4. fail2ban watching ssh sudo fail2ban-client status sshd | grep -q \u0026#34;Status.*ok\u0026#34; \u0026amp;\u0026amp; echo \u0026#34;fail2ban ok\u0026#34; # 5. docker works without sudo docker run --rm hello-world That\u0026rsquo;s it. No password guessing, no open ports you don\u0026rsquo;t know about, a login audit trail, the five essential tools, and Docker ready to go. From here it\u0026rsquo;s a clean base — add your app\u0026rsquo;s ports to UFW as you deploy, and you\u0026rsquo;re production-ready.\n","permalink":"https://blogs.rameskum.com/posts/linux-server-setup/","summary":"\u003cp\u003eEvery fresh VPS (Hetzner, DigitalOcean, your homelab Proxmox box) boots up\nwith the same problem: a root login over password, no firewall, and nothing\ninstalled. Before you deploy anything, spend 15 minutes on this checklist.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eOrder matters.\u003c/strong\u003e Firewall rules and SSH hardening can lock you out\npermanently if done in the wrong sequence. Do the steps in this order and you\nwon\u0026rsquo;t have to rescue-boot your way back in.\u003c/p\u003e\n\u003ch2 id=\"1-update-everything-first\"\u003e1. Update everything first\u003c/h2\u003e\n\u003cp\u003eConnect as root and patch the box before touching anything else.\u003c/p\u003e","title":"New Linux Server? Do These 9 Things Before Anything Else"},{"content":"Rate limiting shows up in two places: your on-call rotation and your system design interview. Both reward the same thing — knowing which algorithm to pick, why, and what breaks at scale.\nThis guide builds three working limiters in Spring Boot, each with tests:\nFixed window — the simplest thing that works, hand-rolled. Token bucket — smooth, burst-friendly, via Bucket4j. Distributed sliding window — precise and shared across instances, on Redis with Lua. Then we close with the part interviews actually grade: how to talk through the tradeoffs.\nThe algorithms in 60 seconds Fixed window: count requests per key in the current window (e.g. 100/minute). Dead simple. Flaw: a burst at the end of one window plus a burst at the start of the next lets through 2x the limit. Sliding window log: store a timestamp per request, evict old ones, count what remains. Precise. Flaw: memory grows with request volume. Token bucket: tokens refill at a steady rate; each request spends one. Allows bursts up to the bucket capacity while keeping the average rate. The best default for APIs. Leaky bucket: like token bucket but requests queue and drain at a constant rate. Smooth output, adds latency under load. For most services, token bucket is the answer. For \u0026ldquo;exactly N requests per rolling window\u0026rdquo; guarantees, sliding window on Redis is the answer. Fixed window is the answer when you want something simple on a single instance and can tolerate the boundary burst.\nBuild 1: fixed window, hand-rolled Start with the simplest limiter so the moving parts are visible. Inject a Clock so tests can control time without sleeping.\npublic class FixedWindowRateLimiter { private final int maxRequests; private final Duration windowSize; private final Clock clock; private final ConcurrentHashMap\u0026lt;String, Window\u0026gt; windows = new ConcurrentHashMap\u0026lt;\u0026gt;(); private record Window(long windowStartMillis, AtomicInteger count) {} public FixedWindowRateLimiter(int maxRequests, Duration windowSize) { this(maxRequests, windowSize, Clock.systemUTC()); } FixedWindowRateLimiter(int maxRequests, Duration windowSize, Clock clock) { this.maxRequests = maxRequests; this.windowSize = windowSize; this.clock = clock; } public boolean tryAcquire(String key) { long now = clock.millis(); Window window = windows.compute(key, (k, existing) -\u0026gt; { if (existing == null || now - existing.windowStartMillis() \u0026gt;= windowSize.toMillis()) { return new Window(now, new AtomicInteger(1)); } existing.count().incrementAndGet(); return existing; }); return window.count().get() \u0026lt;= maxRequests; } } Wire it into a filter. The key can be the client IP, an API key, or the authenticated user id — pick per endpoint.\n@Component public class RateLimitFilter extends OncePerRequestFilter { private final FixedWindowRateLimiter limiter = new FixedWindowRateLimiter(100, Duration.ofMinutes(1)); @Override protected void doFilterInternal(HttpServletRequest request, HttpServletResponse response, FilterChain chain) throws ServletException, IOException { String key = request.getRemoteAddr(); if (!limiter.tryAcquire(key)) { response.setStatus(HttpStatus.TOO_MANY_REQUESTS.value()); response.setHeader(\u0026#34;Retry-After\u0026#34;, \u0026#34;60\u0026#34;); response.setContentType(MediaType.APPLICATION_JSON_VALUE); response.getWriter().write(\u0026#34;{\\\u0026#34;error\\\u0026#34;:\\\u0026#34;rate limit exceeded\\\u0026#34;}\u0026#34;); return; } chain.doFilter(request, response); } } Two things to notice. The 429 status plus Retry-After header is the standard contract — clients and interviewers both expect it. And the map grows with distinct keys, so in production you need eviction (a Cache from Caffeine with expireAfterAccess is the usual fix) or you have a slow memory leak keyed by attacker-controlled input.\nTest it with a controllable clock — no Thread.sleep in tests:\nclass FixedWindowRateLimiterTest { private final AtomicReference\u0026lt;Instant\u0026gt; now = new AtomicReference\u0026lt;\u0026gt;(Instant.parse(\u0026#34;2026-09-20T12:00:00Z\u0026#34;)); private final Clock clock = new Clock() { @Override public ZoneId getZone() { return ZoneOffset.UTC; } @Override public Clock withZone(ZoneId zone) { return this; } @Override public Instant instant() { return now.get(); } }; private FixedWindowRateLimiter limiter; @BeforeEach void setUp() { limiter = new FixedWindowRateLimiter(5, Duration.ofMinutes(1), clock); } @Test void allowsUpToLimitWithinWindow() { for (int i = 0; i \u0026lt; 5; i++) { assertTrue(limiter.tryAcquire(\u0026#34;user-1\u0026#34;)); } assertFalse(limiter.tryAcquire(\u0026#34;user-1\u0026#34;)); } @Test void resetsWhenWindowExpires() { for (int i = 0; i \u0026lt; 5; i++) limiter.tryAcquire(\u0026#34;user-1\u0026#34;); now.set(now.get().plusSeconds(61)); assertTrue(limiter.tryAcquire(\u0026#34;user-1\u0026#34;)); } @Test void tracksKeysIndependently() { for (int i = 0; i \u0026lt; 5; i++) limiter.tryAcquire(\u0026#34;user-1\u0026#34;); assertFalse(limiter.tryAcquire(\u0026#34;user-1\u0026#34;)); assertTrue(limiter.tryAcquire(\u0026#34;user-2\u0026#34;)); } } Build 2: token bucket with Bucket4j For a burst-friendly limiter, reach for Bucket4j instead of hand-rolling refill math:\n\u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;com.bucket4j\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;bucket4j_jdk17-core\u0026lt;/artifactId\u0026gt; \u0026lt;version\u0026gt;8.20.0\u0026lt;/version\u0026gt; \u0026lt;/dependency\u0026gt; One bucket per key, built lazily. The configuration below reads as \u0026ldquo;capacity 100, refill 100 tokens per minute, greedily\u0026rdquo;:\n@Component public class Bucket4jRateLimiter { private final ConcurrentHashMap\u0026lt;String, Bucket\u0026gt; buckets = new ConcurrentHashMap\u0026lt;\u0026gt;(); public boolean tryConsume(String key) { return buckets.computeIfAbsent(key, this::newBucket).tryConsume(1); } private Bucket newBucket(String key) { return Bucket.builder() .addLimit(limit -\u0026gt; limit .capacity(100) .refillGreedy(100, Duration.ofMinutes(1))) .build(); } } The same filter shape from Build 1 works — swap the limiter. Bucket4j also ships a Spring Boot starter (com.giffing.bucket4j.spring.boot.starter) if you prefer annotation-driven config, and a distributed mode backed by Redis or Postgres via its proxy manager.\nTest the capacity behavior without time control — exhausting a bucket needs no clock:\nclass Bucket4jRateLimiterTest { private final Bucket4jRateLimiter limiter = new Bucket4jRateLimiter(); @Test void allowsBurstUpToCapacityThenRejects() { for (int i = 0; i \u0026lt; 100; i++) { assertTrue(limiter.tryConsume(\u0026#34;user-1\u0026#34;)); } assertFalse(limiter.tryConsume(\u0026#34;user-1\u0026#34;)); } @Test void bucketsArePerKey() { for (int i = 0; i \u0026lt; 100; i++) limiter.tryConsume(\u0026#34;user-1\u0026#34;); assertFalse(limiter.tryConsume(\u0026#34;user-1\u0026#34;)); assertTrue(limiter.tryConsume(\u0026#34;user-2\u0026#34;)); } } Build 3: distributed sliding window on Redis In-memory limiters are per-instance. With three replicas behind a load balancer, a client gets 3x your limit. Share state in Redis. To make the check-and-record step atomic — and avoid the race between \u0026ldquo;check count\u0026rdquo; and \u0026ldquo;record request\u0026rdquo; — run a Lua script on the Redis server.\nThe script keeps a sorted set of request timestamps per key, evicts entries outside the window, and only records the request if the count is under the limit:\n-- src/main/resources/lua/sliding-window.lua -- KEYS[1] = rate limit key -- ARGV[1] = now (millis), ARGV[2] = window (millis), -- ARGV[3] = limit, ARGV[4] = unique member local windowStart = tonumber(ARGV[1]) - tonumber(ARGV[2]) redis.call(\u0026#39;ZREMRANGEBYSCORE\u0026#39;, KEYS[1], 0, windowStart) local count = redis.call(\u0026#39;ZCARD\u0026#39;, KEYS[1]) if count \u0026lt; tonumber(ARGV[3]) then redis.call(\u0026#39;ZADD\u0026#39;, KEYS[1], ARGV[1], ARGV[4]) redis.call(\u0026#39;PEXPIRE\u0026#39;, KEYS[1], ARGV[2]) return 1 end return 0 @Component public class RedisSlidingWindowLimiter { private final StringRedisTemplate redis; private final DefaultRedisScript\u0026lt;Long\u0026gt; script; public RedisSlidingWindowLimiter(StringRedisTemplate redis) { this.redis = redis; this.script = new DefaultRedisScript\u0026lt;\u0026gt;(); this.script.setLocation(new ClassPathResource(\u0026#34;lua/sliding-window.lua\u0026#34;)); this.script.setResultType(Long.class); } public boolean tryAcquire(String key, int limit, Duration window) { long now = System.currentTimeMillis(); String member = now + \u0026#34;:\u0026#34; + UUID.randomUUID(); Long allowed = redis.execute(script, List.of(\u0026#34;ratelimit:\u0026#34; + key), String.valueOf(now), String.valueOf(window.toMillis()), String.valueOf(limit), member); return allowed != null \u0026amp;\u0026amp; allowed == 1L; } } Why this shape matters in an interview: the ZREMRANGEBYSCORE + ZCARD + ZADD sequence must be atomic or two concurrent requests can both read a count under the limit and both proceed. Lua gives you that atomicity without a distributed lock. The PEXPIRE keeps dead keys from accumulating.\nFor the test, spin up real Redis with Testcontainers — Lua behavior is exactly what you are testing, so do not mock it:\n@Testcontainers class RedisSlidingWindowLimiterTest { @Container static GenericContainer\u0026lt;?\u0026gt; redis = new GenericContainer\u0026lt;\u0026gt;(\u0026#34;redis:7-alpine\u0026#34;).withExposedPorts(6379); private RedisSlidingWindowLimiter limiter; @BeforeEach void setUp() { LettuceConnectionFactory factory = new LettuceConnectionFactory( redis.getHost(), redis.getMappedPort(6379)); factory.afterPropertiesSet(); limiter = new RedisSlidingWindowLimiter(new StringRedisTemplate(factory)); } @Test void enforcesLimitAcrossCalls() { for (int i = 0; i \u0026lt; 5; i++) { assertTrue(limiter.tryAcquire(\u0026#34;user-1\u0026#34;, 5, Duration.ofMinutes(1))); } assertFalse(limiter.tryAcquire(\u0026#34;user-1\u0026#34;, 5, Duration.ofMinutes(1))); assertTrue(limiter.tryAcquire(\u0026#34;user-2\u0026#34;, 5, Duration.ofMinutes(1))); } } Externalize the configuration Hardcoded limits get stale. Bind them to properties so each environment — and each endpoint — can differ:\nrate-limit: requests-per-minute: 100 window: 1m @ConfigurationProperties(prefix = \u0026#34;rate-limit\u0026#34;) public record RateLimitProperties(int requestsPerMinute, Duration window) {} How to talk through this in an interview The code gets you in the door; the reasoning gets you the offer. Interviewers grade the discussion, so structure it:\n1. Clarify the dimension. Per user, per IP, per API key, per endpoint, or global? Sustained rate, burst allowance, or both? \u0026ldquo;100 requests per minute per user\u0026rdquo; and \u0026ldquo;10,000 requests per second globally\u0026rdquo; are different systems.\n2. Propose with tradeoffs, not just a name. \u0026ldquo;I\u0026rsquo;d start with token bucket: it allows short bursts, which real clients produce, while bounding the average. Fixed window is simpler but lets through 2x at the boundary — here\u0026rsquo;s the scenario.\u0026rdquo; Draw the boundary-burst case; it is the classic follow-up.\n3. Go distributed before they ask. \u0026ldquo;In-memory state doesn\u0026rsquo;t survive multiple replicas. I\u0026rsquo;d share counters in Redis, and use a Lua script so check-and-increment is atomic.\u0026rdquo; Mention that sticky sessions are an alternative but push the complexity into routing.\n4. Name the failure modes. What happens when Redis is down — fail open (availability) or fail closed (protection)? The answer depends on the endpoint: fail closed for payments, fail open for read APIs. Mention clock skew (use Redis server time via TIME if paranoid), hot keys hammering one Redis slot, and memory growth from unbounded keys.\n5. Cover the contract. 429 Too Many Requests, Retry-After, and the X-RateLimit-Limit / X-RateLimit-Remaining / X-RateLimit-Reset headers. Rate limits your clients can\u0026rsquo;t see will generate support tickets.\n6. Say where it runs. API gateway (Spring Cloud Gateway\u0026rsquo;s RequestRateLimiter) for coarse global limits, in-service filter for per-user or per-endpoint rules. Both, usually.\nProduction checklist Enforce at the gateway for global abuse protection and in the service for business rules. Return 429 with Retry-After and the X-RateLimit-* headers on every rejection. Emit a metric on rejections (http.server.requests tagged by outcome, or a dedicated counter) and alert on spikes — a sudden wall of 429s is either an attack or a misconfigured client. Size Redis for the key count: sliding-window logs store one entry per request inside the window. At high volume, consider the sliding-window counter approximation instead. Bound in-memory key sets with eviction; never let attacker-controlled keys grow a map forever. Document the limits for your API consumers. The kindest rate limiter is one nobody is surprised by. ","permalink":"https://blogs.rameskum.com/posts/spring-boot-rate-limiting/","summary":"\u003cp\u003eRate limiting shows up in two places: your on-call rotation and your system design interview. Both reward the same thing — knowing which algorithm to pick, why, and what breaks at scale.\u003c/p\u003e\n\u003cp\u003eThis guide builds three working limiters in Spring Boot, each with tests:\u003c/p\u003e\n\u003col\u003e\n\u003cli\u003e\u003cstrong\u003eFixed window\u003c/strong\u003e — the simplest thing that works, hand-rolled.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eToken bucket\u003c/strong\u003e — smooth, burst-friendly, via Bucket4j.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eDistributed sliding window\u003c/strong\u003e — precise and shared across instances, on Redis with Lua.\u003c/li\u003e\n\u003c/ol\u003e\n\u003cp\u003eThen we close with the part interviews actually grade: how to talk through the tradeoffs.\u003c/p\u003e","title":"Rate Limiting in Spring Boot: The System Design Interview Answer, in Code"},{"content":"Virtual threads can be one of the easiest throughput wins in a blocking Spring Boot application. The switch is one line:\nspring: threads: virtual: enabled: true But that line changes the concurrency model of the application. It does not make database queries faster, reduce API latency, or create more database connections. It makes waiting cheaper by allowing many request tasks to share a smaller number of carrier threads.\nThat is excellent for I/O-heavy services. It also means an old Tomcat worker-thread limit may no longer provide the backpressure you thought it did.\nThis guide builds a small endpoint, verifies that requests really use virtual threads, adds an explicit downstream concurrency limit, and finishes with a production checklist.\nWhat virtual threads improve A platform thread normally occupies an operating-system thread while it exists. A virtual thread is scheduled by the JVM and can unmount from its carrier while waiting on supported blocking I/O. The carrier can then run another virtual thread.\nThe important distinction is:\nVirtual threads improve scale, not speed. One database query does not complete faster merely because it runs on a virtual thread. They fit blocking, I/O-heavy code. Spring MVC, JDBC, JPA, blocking HTTP calls, and file or socket I/O are natural candidates. They do not accelerate CPU-heavy work. More runnable threads cannot create more CPU cores. They should not be pooled. The intended model is one new virtual thread per task. OpenJDK describes virtual threads as a way to raise throughput while preserving the familiar thread-per-request programming model. They are not \u0026ldquo;faster threads.\u0026rdquo;\nPrerequisites Use:\nJava 21 or newer Spring Boot with virtual-thread support (spring.threads.virtual.enabled was introduced in the Spring Boot 3.2 generation) Spring MVC with an embedded servlet container for this example The sample needs Spring Web and, optionally, Actuator:\n\u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;org.springframework.boot\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-boot-starter-web\u0026lt;/artifactId\u0026gt; \u0026lt;/dependency\u0026gt; \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;org.springframework.boot\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-boot-starter-actuator\u0026lt;/artifactId\u0026gt; \u0026lt;/dependency\u0026gt; \u0026lt;dependency\u0026gt; \u0026lt;groupId\u0026gt;org.springframework.boot\u0026lt;/groupId\u0026gt; \u0026lt;artifactId\u0026gt;spring-boot-starter-test\u0026lt;/artifactId\u0026gt; \u0026lt;scope\u0026gt;test\u0026lt;/scope\u0026gt; \u0026lt;/dependency\u0026gt; Then enable virtual threads:\nspring: threads: virtual: enabled: true management: endpoints: web: exposure: include: health,metrics For a normal web application, the embedded server keeps the JVM alive. For a non-web application that relies only on virtual-thread scheduled work, remember that virtual threads are daemon threads. Review your application\u0026rsquo;s lifecycle and keep-alive behavior before deploying.\nProve the switch is active Do not trust configuration alone. Expose a temporary diagnostic endpoint and inspect the thread handling a real HTTP request.\npackage com.example.loomdemo; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; @RestController @RequestMapping(\u0026#34;/demo\u0026#34;) class VirtualThreadController { @GetMapping(\u0026#34;/thread\u0026#34;) ThreadReport thread() { Thread current = Thread.currentThread(); return new ThreadReport(current.toString(), current.isVirtual()); } @GetMapping(\u0026#34;/io\u0026#34;) ThreadReport simulatedIo( @RequestParam(defaultValue = \u0026#34;250\u0026#34;) long delayMs ) throws InterruptedException { if (delayMs \u0026lt; 0 || delayMs \u0026gt; 5_000) { throw new IllegalArgumentException(\u0026#34;delayMs must be between 0 and 5000\u0026#34;); } Thread.sleep(delayMs); Thread current = Thread.currentThread(); return new ThreadReport(current.toString(), current.isVirtual()); } record ThreadReport(String thread, boolean virtual) {} } Run the application and call it:\ncurl -s http://localhost:8080/demo/thread The response should contain:\n{\u0026#34;thread\u0026#34;:\u0026#34;VirtualThread[...]\u0026#34;,\u0026#34;virtual\u0026#34;:true} The exact thread text is JVM-specific. The stable assertion is virtual: true.\nYou can also verify it through an integration test that reaches the actual embedded server. MockMvc is not suitable for this particular check because the test can execute controller code on the test thread rather than through Tomcat.\npackage com.example.loomdemo; import static org.assertj.core.api.Assertions.assertThat; import java.net.URI; import java.net.http.HttpClient; import java.net.http.HttpRequest; import java.net.http.HttpResponse; import org.junit.jupiter.api.Test; import org.springframework.boot.test.context.SpringBootTest; import org.springframework.boot.test.web.server.LocalServerPort; @SpringBootTest(webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT) class VirtualThreadSmokeTest { @LocalServerPort int port; @Test void tomcatHandlesTheRequestOnAVirtualThread() throws Exception { HttpRequest request = HttpRequest.newBuilder() .uri(URI.create(\u0026#34;http://localhost:\u0026#34; + port + \u0026#34;/demo/thread\u0026#34;)) .GET() .build(); HttpResponse\u0026lt;String\u0026gt; response = HttpClient.newHttpClient().send( request, HttpResponse.BodyHandlers.ofString() ); assertThat(response.statusCode()).isEqualTo(200); assertThat(response.body()).contains(\u0026#34;\\\u0026#34;virtual\\\u0026#34;:true\u0026#34;); } } Remove the public diagnostic endpoint after validation, or protect it as an internal-only endpoint.\nThe server.tomcat.threads.max trap A traditional Tomcat connector uses a bounded worker pool. In that model, this property is meaningful:\nserver: tomcat: threads: max: 200 With virtual request threads enabled, Tomcat executes tasks using a virtual-thread-per-task executor. There is no reusable request-worker pool with 200 virtual threads waiting inside it. As a result, server.tomcat.threads.max is not a cap on request concurrency in this mode.\nThat difference is easy to miss because the property still looks valid in configuration. The application starts, but that number no longer limits concurrency the way a bounded platform-thread pool used to.\nDo not replace it with \u0026ldquo;a bigger virtual-thread pool.\u0026rdquo; Virtual threads are intentionally not pooled. Put limits around the scarce resource instead:\ndatabase connections outbound calls to a vendor API Kafka producer or consumer work filesystem or object-storage operations memory-heavy request processing Transport settings such as connection limits and accept queues solve a different problem. They do not express how much concurrent work a database or downstream service can safely handle.\nAdd explicit backpressure Assume a payment provider allows only 40 calls from this service at once. A semaphore makes that limit visible and independent of the web-server thread model.\npackage com.example.loomdemo; import static java.util.concurrent.TimeUnit.MILLISECONDS; import static org.springframework.http.HttpStatus.BAD_GATEWAY; import static org.springframework.http.HttpStatus.SERVICE_UNAVAILABLE; import static org.springframework.http.HttpStatus.TOO_MANY_REQUESTS; import java.util.concurrent.Callable; import java.util.concurrent.Semaphore; import org.springframework.stereotype.Component; import org.springframework.web.server.ResponseStatusException; @Component class DownstreamGuard { private final Semaphore permits = new Semaphore(40); \u0026lt;T\u0026gt; T call(Callable\u0026lt;T\u0026gt; work) { boolean acquired = false; try { acquired = permits.tryAcquire(250, MILLISECONDS); if (!acquired) { throw new ResponseStatusException( TOO_MANY_REQUESTS, \u0026#34;Too many concurrent downstream calls\u0026#34; ); } return work.call(); } catch (InterruptedException exception) { Thread.currentThread().interrupt(); throw new ResponseStatusException( SERVICE_UNAVAILABLE, \u0026#34;Request interrupted\u0026#34;, exception ); } catch (ResponseStatusException exception) { throw exception; } catch (Exception exception) { throw new ResponseStatusException( BAD_GATEWAY, \u0026#34;Downstream call failed\u0026#34;, exception ); } finally { if (acquired) { permits.release(); } } } } Use the guard at the integration boundary. This endpoint simulates a protected downstream call without introducing another dependency:\npackage com.example.loomdemo; import java.util.Map; import org.springframework.web.bind.annotation.GetMapping; import org.springframework.web.bind.annotation.RequestParam; import org.springframework.web.bind.annotation.RestController; @RestController class ProtectedIoController { private final DownstreamGuard guard; ProtectedIoController(DownstreamGuard guard) { this.guard = guard; } @GetMapping(\u0026#34;/demo/protected-io\u0026#34;) Map\u0026lt;String, Object\u0026gt; protectedIo( @RequestParam(defaultValue = \u0026#34;250\u0026#34;) long delayMs ) { return guard.call(() -\u0026gt; { Thread.sleep(delayMs); return Map.of( \u0026#34;virtual\u0026#34;, Thread.currentThread().isVirtual(), \u0026#34;delayMs\u0026#34;, delayMs ); }); } } A Semaphore parks a waiting virtual thread without requiring a dedicated platform thread. Set the permit count from the real capacity of the dependency, not from the number of virtual threads the JVM can create.\nFor JDBC or JPA, the connection pool is already a concurrency boundary. Do not multiply the pool size merely because the application can create more request threads. Watch connection-acquisition time, pool saturation, database CPU, lock waits, and query latency first.\nThe pinning advice depends on your JDK Many virtual-thread guides say to replace every synchronized block with ReentrantLock. That advice needs a version label.\nJava 21 through Java 23 A virtual thread that blocks while inside a synchronized method or block can pin its carrier. Frequent, long-lived pinning can reduce scalability. This pattern deserves attention:\nsynchronized (lock) { return remoteClient.fetch(); // blocking I/O while holding a monitor } On these JDKs, moving a hot blocking section to a ReentrantLock, or redesigning it so the I/O happens outside the critical section, can prevent that pinning.\nJava 24 and newer JEP 491 changed monitor handling so a virtual thread can unmount while blocked in synchronized code. Do not mechanically rewrite every monitor just for virtual-thread compatibility on these JDKs.\nThe design rule still matters: keep critical sections narrow and avoid slow I/O while holding locks when practical. That reduces contention regardless of thread type.\nSome uncommon pinning cases remain, including blocking through certain native or foreign-function call paths. On modern JDKs, use JFR instead of the old -Djdk.tracePinnedThreads switch. JEP 491 made that switch unnecessary but kept the jdk.VirtualThreadPinned event for the remaining cases.\nStart a short recording against a representative workload:\nPID=$(pgrep -f \u0026#39;java.*app.jar\u0026#39; | head -1) jcmd \u0026#34;$PID\u0026#34; JFR.start \\ name=virtual-threads \\ settings=profile \\ duration=60s \\ filename=virtual-threads.jfr Open the recording in JDK Mission Control and inspect virtual-thread pinning events, socket waits, allocation pressure, and the code paths consuming CPU.\nAudit ThreadLocal usage Virtual threads support ThreadLocal, so most existing libraries continue to work. The risk is scale: data stored once per thread can be multiplied across a much larger number of short-lived threads.\nLook especially for code using a thread local as a resource cache:\nprivate static final ThreadLocal\u0026lt;ExpensiveClient\u0026gt; CLIENT = ThreadLocal.withInitial(ExpensiveClient::new); That pattern made more sense when a small pool reused the same worker threads. It is a poor fit for one-thread-per-task execution because every task can initialize another expensive object.\nPrefer a properly bounded shared client or pool managed by the framework. Request-scoped metadata such as trace IDs can still use supported context propagation, but measure allocation and retained memory under realistic concurrency.\nOn JDK 21, this diagnostic option prints a stack trace when a virtual thread sets a thread-local value:\njava -Djdk.traceVirtualThreadLocals=true -jar app.jar Use it in a test environment; the output can be noisy.\nMeasure the right thing Do not benchmark only a /hello endpoint. A no-op handler measures routing and serialization, not the waiting behavior virtual threads are designed to improve.\nStart with the simulated I/O endpoint:\n# 2,000 requests, up to 200 in flight seq 2000 | xargs -P200 -I{} \\ curl -s -o /dev/null -w \u0026#39;%{http_code}\\n\u0026#39; \\ \u0026#39;http://localhost:8080/demo/io?delayMs=250\u0026#39; \\ | sort | uniq -c Run the same workload twice:\n# Run A spring: threads: virtual: enabled: false # Run B spring: threads: virtual: enabled: true Warm up the JVM before recording results. Keep hardware, heap settings, traffic shape, and downstream capacity identical. Then compare:\nthroughput p50, p95, and p99 latency error and timeout rate CPU utilization heap usage and allocation rate database-pool wait time outbound-client pool saturation JFR pinning events The expected outcome is not \u0026ldquo;every request gets faster.\u0026rdquo; For a sufficiently concurrent, waiting-heavy workload, the service can keep more requests in progress without consuming one operating-system thread per request. If CPU is already saturated, or the database is the bottleneck, virtual threads may produce little improvement and can expose the downstream limit sooner.\nReview custom executors The global switch configures eligible Spring Boot auto-configured execution paths. It does not magically replace every executor created by application code or a library.\nSearch the codebase for:\nExecutors.newFixedThreadPool Executors.newCachedThreadPool ThreadPoolTaskExecutor ThreadPoolTaskScheduler @Async CompletableFuture.supplyAsync parallelStream For each result, answer:\nIs the work CPU-bound or mostly waiting on I/O? Which executor actually runs it? Is its queue bounded? Was its pool size acting as backpressure? What happens during shutdown and cancellation? Do not convert CPU-bound executors to virtual threads just for consistency. A bounded platform-thread executor is often the right choice for computational work.\nSpring Boot also notes that pooling properties are ignored when its virtual-thread scheduler is active. If an application depends on scheduler pool size to restrict concurrent jobs, introduce an explicit concurrency limit in the job itself.\nProduction checklist Before enabling the flag in production:\nConfirm the runtime: use Java 21 or newer and record the exact JDK version in deployment metadata. Verify a real request: assert Thread.currentThread().isVirtual() through the embedded server. Inventory custom executors: the global property does not prove every task uses the same execution model. Replace accidental backpressure: do not rely on server.tomcat.threads.max to limit virtual request concurrency. Protect dependencies: size semaphores, bulkheads, and client pools from actual downstream capacity. Review database pressure: inspect connection-pool waits, query latency, locks, and database CPU. Audit thread locals: remove per-thread caches of expensive resources and measure memory under load. Apply version-correct pinning advice: audit synchronized blocking paths on Java 21–23; on Java 24+, focus on the remaining JFR events and general lock contention. Load test realistic I/O: include representative database and HTTP latency, timeouts, and failure behavior. Compare tail latency and errors: throughput alone can hide overloaded dependencies. Test shutdown: verify scheduled tasks, async work, and graceful termination. Roll out gradually: use a canary, observe it, and keep a fast rollback path. Final takeaway spring.threads.virtual.enabled=true is simple; operating it safely requires a clear concurrency model.\nUse virtual threads when the service handles many concurrent tasks that spend substantial time waiting. Do not expect lower latency from the switch alone. Make downstream limits explicit, verify the actual execution thread, audit thread-local resource caches, and interpret pinning advice according to the JDK version you run.\nThe best migration is not \u0026ldquo;turn on virtual threads everywhere.\u0026rdquo; It is \u0026ldquo;make waiting cheap while keeping scarce resources bounded.\u0026rdquo;\nSources JEP 444: Virtual Threads JEP 491: Synchronize Virtual Threads without Pinning Spring Boot: Task Execution and Scheduling Spring Boot ConditionalOnVirtualThreads API Apache Tomcat VirtualThreadExecutor ","permalink":"https://blogs.rameskum.com/posts/spring-boot-virtual-threads/","summary":"\u003cp\u003eVirtual threads can be one of the easiest throughput wins in a blocking Spring Boot application. The switch is one line:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-yaml\" data-lang=\"yaml\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"nt\"\u003espring\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"nt\"\u003ethreads\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"nt\"\u003evirtual\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e      \u003c/span\u003e\u003cspan class=\"nt\"\u003eenabled\u003c/span\u003e\u003cspan class=\"p\"\u003e:\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kc\"\u003etrue\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eBut that line changes the concurrency model of the application. It does not make database queries faster, reduce API latency, or create more database connections. It makes waiting cheaper by allowing many request tasks to share a smaller number of carrier threads.\u003c/p\u003e","title":"Flipping spring.threads.virtual.enabled=true? Read This First"},{"content":"JAVA Streams Sum of Elements Given a list of integers, compute the sum of all numbers.\n// using sum List\u0026lt;Integer\u0026gt; arr = Arrays.asList(1, 4, 5, 6, 22, 3, 90, 89, 2, 1, 3, 4, 55, 6); int sum = arr.stream() .mapToInt(Integer::intValue) .sum(); // or int sum3 = arr.stream() .reduce((a,b) -\u0026gt; a + b) .get(); // or int sum2 = arr.stream() .reduce(0, (total, n) -\u0026gt; total + n, (total1, total2) -\u0026gt; total1 + total2); Average of Elements Given a list of integers, compute the average of all numbers.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(1, 4, 5, 6, 22, 3, 90, 89, 2, 1, 3, 4, 55, 6); // average of all numbers double avg = arr.stream() .mapToInt(n -\u0026gt; n) .average() .orElse(0); Square of All Numbers Given a list of numbers, get the average of squares of all numbers whose square is \u0026gt; 100.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(1, 4, 5, 6, 22, 3, 90, 89, 2, 1, 3, 4, 55, 6); double avg = arr.stream() .mapToInt(n -\u0026gt; n * n) .filter(n -\u0026gt; n \u0026gt; 100) .average() .orElseGet(() -\u0026gt; 0.0); Partition Numbers Given a list of numbers, separate even and odd numbers.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(11, 2, 3, 45, 67, 9, 90, 87, 8, 2); Map\u0026lt;Boolean, List\u0026lt;Integer\u0026gt;\u0026gt; collect = arr.stream() .collect(Collectors.partitioningBy(n -\u0026gt; n % 2 == 0)); // or mapping Map\u0026lt;String, List\u0026lt;Integer\u0026gt;\u0026gt; listMap = arr.stream() .collect(groupingBy(n -\u0026gt; n % 2 == 0 ? \u0026#34;EVEN\u0026#34; : \u0026#34;ODD\u0026#34;, toList())); Print Numbers Starts With Prefix 2 Given a list of numbers, print numbers starting with 2.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(11, 2, 3, 45, 67, 9, 90, 87, 8, 2, 22, 201, 302, 1222); List\u0026lt;Integer\u0026gt; startingWith2 = arr.stream() .map(String::valueOf) .filter(str -\u0026gt; str.startsWith(\u0026#34;2\u0026#34;)) .map(Integer::valueOf) .toList(); // or, Solution is the class with startWith2 function List\u0026lt;Integer\u0026gt; startingWith2 = arr.stream() .filter(Solution::startWith2) .toList(); private static boolean startWith2(int n) { int temp = n; while (temp \u0026gt; 0 \u0026amp;\u0026amp; temp != 2) { temp /= 10; } return temp == 2; } Print Duplicate Numbers using Streams Given a list of numbers, find all the duplicate numbers.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(11, 2, 3, 45, 67, 9, 90, 87, 8, 2, 22, 201, 302, 1222); Set\u0026lt;Integer\u0026gt; ele = arr.stream() .filter(n -\u0026gt; Collections.frequency(arr, n) \u0026gt; 1) .collect(Collectors.toSet()); // or good solution would be List\u0026lt;Integer\u0026gt; duplicates = arr.stream() .collect(groupingBy(n -\u0026gt; n, counting())) .entrySet() .stream() .filter(entry -\u0026gt; entry.getValue() \u0026gt; 1) .map(Entry::getKey) .toList(); // or List\u0026lt;Integer\u0026gt; duplicates2 = arr.stream() .collect(Collectors.collectingAndThen( Collectors.groupingBy(e -\u0026gt; e, Collectors.counting()), map -\u0026gt; map.entrySet().stream() .filter(entry -\u0026gt; entry.getValue() \u0026gt; 1) .map(Map.Entry::getKey) .collect(Collectors.toList()) )); Find Max and Min Numbers using Streams Get the minimum value of the list.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(11, -2, 3, 45, 67, 9, 90, 90, 87, 8, 2, 22, 201, 302, 1222); int min = arr.stream() .min(Comparator.comparing(Integer::intValue)) .get(); Get the sum of first 5 numbers Given the list of numbers, find the sum of the first 5 numbers.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(11, -2, 3, 45, 67, 9, 90, 90, 87, 8, 2, 22, 201, 302, 1222); int sum = arr.stream() .limit(5) .mapToInt(n -\u0026gt; n) .sum(); // or int sum2 = arr.stream() .limit(5) .reduce(0, Integer::sum); Second Lowest Smallest Number Find the second smallest number in the stream.\nList\u0026lt;Integer\u0026gt; arr = Arrays.asList(11, -2, 3, 45, 67, 9, 90, 90, 87, 8, 1, 0, 2, 22, 201, 302, 1222); Integer secondSmallest = arr.stream() .sorted() .skip(1) .findFirst() .orElseGet(() -\u0026gt; Integer.MAX_VALUE); ","permalink":"https://blogs.rameskum.com/posts/java-streams/","summary":"\u003ch2 id=\"java-streams\"\u003eJAVA Streams\u003c/h2\u003e\n\u003ch3 id=\"sum-of-elements\"\u003eSum of Elements\u003c/h3\u003e\n\u003cp\u003eGiven a list of integers, compute the sum of all numbers.\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e// using sum\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eList\u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026lt;\u003c/span\u003e\u003cspan class=\"n\"\u003eInteger\u003c/span\u003e\u003cspan class=\"o\"\u003e\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003earr\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eArrays\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003easList\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e4\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e5\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e6\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e22\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e90\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e89\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e2\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e3\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e4\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e55\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003e6\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003esum\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003earr\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003estream\u003c/span\u003e\u003cspan class=\"p\"\u003e()\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e            \u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003emapToInt\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003eInteger\u003c/span\u003e\u003cspan class=\"p\"\u003e::\u003c/span\u003e\u003cspan class=\"n\"\u003eintValue\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e            \u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003esum\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e// or\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003esum3\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003earr\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003estream\u003c/span\u003e\u003cspan class=\"p\"\u003e()\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e            \u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003ereduce\u003c/span\u003e\u003cspan class=\"p\"\u003e((\u003c/span\u003e\u003cspan class=\"n\"\u003ea\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"n\"\u003eb\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ea\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e+\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eb\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e            \u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eget\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"c1\"\u003e// or\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kt\"\u003eint\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003esum2\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003earr\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003estream\u003c/span\u003e\u003cspan class=\"p\"\u003e()\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003ereduce\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003e0\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e                \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003etotal\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003etotal\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e+\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003en\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e                \u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003etotal1\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003etotal2\u003c/span\u003e\u003cspan class=\"p\"\u003e)\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003etotal1\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e+\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003etotal2\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"average-of-elements\"\u003eAverage of Elements\u003c/h3\u003e\n\u003cp\u003eGiven a list of integers, compute the average of all numbers.\u003c/p\u003e","title":"Java Streams"},{"content":"Thread Thread Subclass Here is an example of creating a Java Thread subclass:\npublic class MyThread extends Thread { public void run(){ System.out.println(\u0026#34;MyThread running\u0026#34;); } } To create and start the above thread you can do like this:\nMyThread myThread = new MyThread(); myTread.start(); You can also create an anonymous subclass of Thread like this:\nThread thread = new Thread(){ public void run(){ System.out.println(\u0026#34;Thread Running\u0026#34;); } } thread.start(); Runnable Interface Implementation Java Class Implements Runnable public class MyRunnable implements Runnable { public void run(){ System.out.println(\u0026#34;MyRunnable running\u0026#34;); } } Anonymous Implementation of Runnable Runnable myRunnable = new Runnable(){ public void run(){ System.out.println(\u0026#34;Runnable running\u0026#34;); } } Java Lambda Implementation of Runnable Runnable runnable = () -\u0026gt; { System.out.println(\u0026#34;Lambda Runnable running\u0026#34;); }; Starting a Thread With a Runnable Runnable runnable = new MyRunnable(); // or an anonymous class, or lambda... Thread thread = new Thread(runnable); thread.start(); The Java Memory Model Here is a diagram illustrating the call stack and local variables stored on the thread stacks, and objects stored on the heap:\nHere is a simplified diagram of modern computer hardware architecture:\nBridging The Gap Between The Java Memory Model And The Hardware Memory Architecture The hardware memory architecture does not distinguish between thread stacks and heap. On the hardware, both the thread stack and the heap are located in main memory.\nWhen objects and variables can be stored in various different memory areas in the computer, certain problems may occur. The two main problems are:\nVisibility of thread updates (writes) to shared variables. Race conditions when reading, checking and writing shared variables. Visibility of Shared Objects If two or more threads are sharing an object, without the proper use of either volatile declarations or synchronization, updates to the shared object made by one thread may not be visible to other threads.\nThe volatile keyword can make sure that a given variable is read directly from main memory, and always written back to main memory when updated.\nRace Conditions If two or more threads share an object, and more than one thread updates variables in that shared object, race conditions may occur.\nSynchronized keyword, blocks and methods in Java Synchronized blocks are reentrant in Java.\nSynchronized blocks can only blocks threads running on same virtual machine.\nLimitations Only ne thread can enter a synchronized block at a time. There is no guarantee about the sequence in which waiting thread gets access to the synchronized block. starvation is possible Performance Overhead Low overhead - when sync block is uncontested (not already locked) Higher overhead - when sync block is contested (already locked by another thread) Synchronized Instance methods Uses MyCounter instance as monitor object.\npublic class MyCounter { private int count = 0; public synchronized void add(int value){ this.count += value; } public synchronized void subtract(int value){ this.count -= value; } } Synchronized Static Methods Uses MyCounter.class as monitor object.\npublic static MyStaticCounter{ private static int count = 0; public static synchronized void add(int value){ count += value; } public static synchronized void subtract(int value){ count -= value; } } Synchronized Blocks in Instance Methods Using instance object as monitor object. public void add(int value){ synchronized(this){ this.count += value; } } The following two examples are both synchronized on the instance they are called on. They are therefore equivalent with respect to synchronization:\npublic class MyClass { public synchronized void log1(String msg1, String msg2){ log.writeln(msg1); log.writeln(msg2); } public void log2(String msg1, String msg2){ synchronized(this){ log.writeln(msg1); log.writeln(msg2); } } } Synchronized Blocks in Static Methods These methods are synchronized on the class object of the class the methods belong to:\npublic class MyClass { public static synchronized void log1(String msg1, String msg2){ log.writeln(msg1); log.writeln(msg2); } public static void log2(String msg1, String msg2){ synchronized(MyClass.class){ log.writeln(msg1); log.writeln(msg2); } } } Synchronized Blocks in Lambda Expressions It is even possible to use synchronized blocks inside a Java Lambda Expression as well as inside anonymous classes.\nimport java.util.function.Consumer; public class SynchronizedExample { public static void main(String[] args) { Consumer\u0026lt;String\u0026gt; func = (String param) -\u0026gt; { synchronized(SynchronizedExample.class) { System.out.println( Thread.currentThread().getName() + \u0026#34; step 1: \u0026#34; + param); try { Thread.sleep( (long) (Math.random() * 1000)); } catch (InterruptedException e) { e.printStackTrace(); } System.out.println( Thread.currentThread().getName() + \u0026#34; step 2: \u0026#34; + param); } }; Thread thread1 = new Thread(() -\u0026gt; { func.accept(\u0026#34;Parameter\u0026#34;); }, \u0026#34;Thread 1\u0026#34;); Thread thread2 = new Thread(() -\u0026gt; { func.accept(\u0026#34;Parameter\u0026#34;); }, \u0026#34;Thread 2\u0026#34;); thread1.start(); thread2.start(); } } Volatile Keyword Use volatile for flags or variables that signal events or state changes between threads. It provides visibility guarantees, but not atomicity (indivisibility) for complex operations. If atomicity is required, consider synchronization mechanisms like synchronized. volatile can improve performance compared to synchronization in scenarios where only visibility is needed. In this below scenario, there\u0026rsquo;s a chance that the consumer thread might keep reading an outdated value of finished from its local CPU cache due to compiler optimizations. This could lead to the consumer waiting indefinitely even though the producer has already set the flag to true.\npublic class TaskCompletion { private boolean finished = false; // Not volatile public void setFinished() { finished = true; // Producer sets the flag } public boolean isFinished() { return finished; // Consumer checks the flag } public static void main(String[] args) { TaskCompletion taskCompletion = new TaskCompletion(); Thread producer = new Thread(() -\u0026gt; taskCompletion.setFinished()); Thread consumer = new Thread(() -\u0026gt; { while (!taskCompletion.isFinished()) { // Busy waiting (inefficient) } System.out.println(\u0026#34;Task completed!\u0026#34;); }); producer.start(); consumer.start(); } } ThreadLocal The Java ThreadLocal class enables you to create variables that can only be read and written by the same thread. Thus, even if two threads are executing the same code, and the code has a reference to the same ThreadLocal variable, the two threads cannot see each other\u0026rsquo;s ThreadLocal variables. Thus, the Java ThreadLocal class provides a simple way to make code thread safe that would not otherwise be so.\nThreadLocal\u0026lt;String\u0026gt; threadLocal = new ThreadLocal\u0026lt;\u0026gt;(); Thread thread1 = new Thread(() -\u0026gt; { threadLocal.set(\u0026#34;Thread 1\u0026#34;); // setting only sets in the current thread System.out.println(threadLocal.get()); threadLocal.remove(); // removing only removes from the current thread System.out.println(threadLocal.get()); }); Thread thread2 = new Thread(() -\u0026gt; { threadLocal.set(\u0026#34;Thread 2\u0026#34;); System.out.println(threadLocal.get()); try { Thread.sleep(1000); } catch (InterruptedException e) { } System.out.println(threadLocal.get()); threadLocal.remove(); System.out.println(threadLocal.get()); }); thread1.start(); thread2.start(); public class MyDateFormatter { private ThreadLocal\u0026lt;SimpleDateFormat\u0026gt; simpleDateFormatThreadLocal = new ThreadLocal\u0026lt;\u0026gt;(); public String format(Date date) { SimpleDateFormat simpleDateFormat = getThreadLocalSimpleDateFormat(); return simpleDateFormat.format(date); } private SimpleDateFormat getThreadLocalSimpleDateFormat() { SimpleDateFormat simpleDateFormat = simpleDateFormatThreadLocal.get(); if(simpleDateFormat == null) { simpleDateFormat = new SimpleDateFormat(\u0026#34;yyyy-MM-dd HH:mm:ss\u0026#34;); simpleDateFormatThreadLocal.set(simpleDateFormat); } return simpleDateFormat; } } Thread Pool A thread pool is a pool threads that can be \u0026ldquo;reused\u0026rdquo; to execute tasks, so that each thread may execute more than one task. A thread pool is an alternative to creating a new thread for each task you need to execute.\nLocks Introduction The Java Lock interface, java.util.concurrent.locks.Lock, represents a concurrent lock which can be used to guard against race conditions inside critical sections.\nclass Example { public static void main(String[] args) { Lock lock = new ReentrantLock(); lock.lock(); // do something lock.unlock(); } } Fail-safe Lock and Unlock Lock lock = new ReentrantLock(); try{ lock.lock(); //critical section } finally { lock.unlock(); } Java ExecutorService ExecutorService executorService = Executors.newFixedThreadPool(10); executorService.execute(new Runnable() { public void run() { System.out.println(\u0026#34;Asynchronous task\u0026#34;); } }); executorService.shutdown(); Java ExecutorService Implementations Since ExecutorService is an interface, you need to its implementations in order to make any use of it. The ExecutorService has the following implementation in the java.util.concurrent package:\nThreadPoolExecutor ScheduledThreadPoolExecutor Creating an ExecutorService ExecutorService executorService1 = Executors.newSingleThreadExecutor(); ExecutorService executorService2 = Executors.newFixedThreadPool(10); ExecutorService executorService3 = Executors.newScheduledThreadPool(10); ","permalink":"https://blogs.rameskum.com/posts/java-concurrency-multithreading/","summary":"\u003ch2 id=\"thread\"\u003eThread\u003c/h2\u003e\n\u003ch3 id=\"thread-subclass\"\u003eThread Subclass\u003c/h3\u003e\n\u003cp\u003eHere is an example of creating a Java \u003ccode\u003eThread\u003c/code\u003e subclass:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kd\"\u003eclass\u003c/span\u003e \u003cspan class=\"nc\"\u003eMyThread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kd\"\u003eextends\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eThread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\t\u003c/span\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003evoid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nf\"\u003erun\u003c/span\u003e\u003cspan class=\"p\"\u003e(){\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\t\t\u003c/span\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;MyThread running\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\t\u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eTo create and start the above thread you can do like this:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eMyThread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003emyThread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eMyThread\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003emyTread\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003estart\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003cp\u003eYou can also create an anonymous subclass of Thread like this:\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eThread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ethread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eThread\u003c/span\u003e\u003cspan class=\"p\"\u003e(){\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\t\u003c/span\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003evoid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nf\"\u003erun\u003c/span\u003e\u003cspan class=\"p\"\u003e(){\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\t\t\u003c/span\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Thread Running\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\t\u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003ethread\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003estart\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"runnable-interface-implementation\"\u003eRunnable Interface Implementation\u003c/h3\u003e\n\u003ch4 id=\"java-class-implements-runnable\"\u003eJava Class Implements Runnable\u003c/h4\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kd\"\u003eclass\u003c/span\u003e \u003cspan class=\"nc\"\u003eMyRunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kd\"\u003eimplements\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eRunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003evoid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nf\"\u003erun\u003c/span\u003e\u003cspan class=\"p\"\u003e(){\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;MyRunnable running\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e  \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch4 id=\"anonymous-implementation-of-runnable\"\u003eAnonymous Implementation of Runnable\u003c/h4\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eRunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003emyRunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eRunnable\u003c/span\u003e\u003cspan class=\"p\"\u003e(){\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"kd\"\u003epublic\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"kt\"\u003evoid\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"nf\"\u003erun\u003c/span\u003e\u003cspan class=\"p\"\u003e(){\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e            \u003c/span\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Runnable running\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e    \u003c/span\u003e\u003cspan class=\"p\"\u003e}\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch4 id=\"java-lambda-implementation-of-runnable\"\u003eJava Lambda Implementation of Runnable\u003c/h4\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eRunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003erunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e        \u003c/span\u003e\u003cspan class=\"p\"\u003e()\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e-\u0026gt;\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Lambda Runnable running\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e};\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch4 id=\"starting-a-thread-with-a-runnable\"\u003eStarting a Thread With a Runnable\u003c/h4\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eRunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003erunnable\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eMyRunnable\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"c1\"\u003e// or an anonymous class, or lambda...\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eThread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ethread\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eThread\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003erunnable\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003ethread\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003estart\u003c/span\u003e\u003cspan class=\"p\"\u003e();\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"the-java-memory-model\"\u003eThe Java Memory Model\u003c/h3\u003e\n\u003cp\u003eHere is a diagram illustrating the call stack and local variables stored on the thread stacks, and objects stored on the heap:\u003c/p\u003e","title":"Java Concurrency Multithreading"},{"content":"Asynchronous Programming in Java Parallel vs. Concurrent vs. Asynchronous Parallel: Multiple tasks executed at the same time. For example, I can walk and talk at the same time. Concurrent: I can work on multiple tasks, but I can do only one task at a moment in time. For example, I can talk or drink water at a moment. Asynchronous: It just means non-blocking. For example, I can brew a coffee, and in the meantime, I could do something else. responsive preemptive performance For Async programming JavaScript used callback. Callbacks created a problems of its own like callback hell. Promise came to solve the problem.\nPromise State of a promise.\nPending when the promise is executed. Resolved when the promise is completed. Rejected when promise failed. Completable Future CompletableFuture in Java is Promises in JavaScript runAsync - triggers the promise and forgets. supplyAsync - get the promise response. thenApply - maps response of promise thenAccept - consume the response of promise get - blocks the thread for promise to resolve or reject thenCompose - if the response is another promise thenCombine - to combine multiple promise import lombok.SneakyThrows; import lombok.extern.slf4j.Slf4j; import java.util.concurrent.CompletableFuture; import java.util.concurrent.ExecutionException; @Slf4j public class Sample { @SneakyThrows public static int compute(int n) { Thread.sleep(1000); return n * 2; } public static CompletableFuture\u0026lt;Integer\u0026gt; create(int n) { return CompletableFuture.supplyAsync(() -\u0026gt; compute(n)); } public static void main(String[] args) throws ExecutionException, InterruptedException { log.info(\u0026#34;future: {}\u0026#34;, create(5)); // Not Complete create(4) .thenApply(data -\u0026gt; data + 1) .thenAccept(r -\u0026gt; log.info(\u0026#34;future: {}\u0026#34;, r)) .get(); var cf1 = create(2); var cf2 = create(3); // thenCombine cf1.thenCombine(cf2, Integer::sum) .thenAccept(v -\u0026gt; log.info(\u0026#34;sum: {}\u0026#34;, v)); // thenCompose create(2) //\t.thenApply(Sample::create) .thenCompose(Sample::create) .thenAccept(r -\u0026gt; log.info(\u0026#34;future: {}\u0026#34;, r)); } } ","permalink":"https://blogs.rameskum.com/posts/aynchronous-programming/","summary":"\u003ch2 id=\"asynchronous-programming-in-java\"\u003eAsynchronous Programming in Java\u003c/h2\u003e\n\u003ch3 id=\"parallel-vs-concurrent-vs-asynchronous\"\u003eParallel vs. Concurrent vs. Asynchronous\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eParallel\u003c/strong\u003e: Multiple tasks executed at the same time. For example, I can walk and talk at the same time.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eConcurrent\u003c/strong\u003e: I can work on multiple tasks, but I can do only one task at a moment in time. For example, I can talk or drink water at a moment.\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eAsynchronous\u003c/strong\u003e: It just means non-blocking. For example, I can brew a coffee, and in the meantime, I could do something else.\n\u003cul\u003e\n\u003cli\u003eresponsive\u003c/li\u003e\n\u003cli\u003epreemptive\u003c/li\u003e\n\u003cli\u003eperformance\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003cblockquote\u003e\n\u003cp\u003eFor Async programming JavaScript used \u003cstrong\u003ecallback\u003c/strong\u003e.\n\u003cstrong\u003eCallbacks\u003c/strong\u003e created a problems of its own like callback hell.\n\u003cstrong\u003ePromise\u003c/strong\u003e came to solve the problem.\u003c/p\u003e","title":"Asynchronous Programming in Java"},{"content":"AWS Use Case Questions You\u0026rsquo;re setting up a website for a small shop using AWS. How would you choose the right AWS tools to make sure the website stays fast and reliable, whether there are only a few visitors or a lot of people shopping at once during a big sale? To set up a website for a small shop using AWS, ensuring it stays fast and reliable regardless of traffic fluctuations, follow these steps:\nDomain Registration and DNS Management\nRoute 53: Use Route 53 to register your domain and manage DNS settings. It’s a scalable and reliable DNS service. Hosting the Website\nAmazon S3: If your website is static (HTML, CSS, JavaScript), host it on Amazon S3. S3 is highly available and can handle sudden traffic spikes. Amazon EC2 or AWS Elastic Beanstalk: For dynamic websites, use EC2 instances to run your application. Alternatively, AWS Elastic Beanstalk simplifies the deployment and management of the application. Content Delivery\nAmazon CloudFront: Use CloudFront as a CDN to distribute content globally with low latency. It caches content at edge locations, speeding up delivery to users. Database Management\nAmazon RDS: For relational databases, use Amazon RDS (MySQL, PostgreSQL, etc.). RDS handles backups, software patching, automatic failure detection, and recovery. Amazon DynamoDB: For NoSQL databases, DynamoDB offers single-digit millisecond performance at any scale. Scalability and Load Balancing\nAuto Scaling: Set up Auto Scaling to automatically adjust the number of EC2 instances based on traffic. Elastic Load Balancing (ELB): Use ELB to distribute incoming traffic across multiple EC2 instances, ensuring no single instance is overwhelmed. Security\nAWS WAF: Deploy AWS Web Application Firewall (WAF) to protect your website from common web exploits. AWS Shield: For additional DDoS protection, use AWS Shield. IAM: Implement AWS Identity and Access Management (IAM) to control access to AWS services and resources securely. Monitoring and Management\nAmazon CloudWatch: Use CloudWatch to monitor your resources and set alarms for specific thresholds (CPU usage, memory usage, etc.). AWS CloudTrail: Enable CloudTrail to log and monitor account activity for security and operational auditing. Backup and Recovery\nAmazon S3/Glacier: Use S3 for regular backups and Glacier for long-term, cost-effective storage of infrequent backups. AWS Backup: Centralize and automate backup across AWS services. CI/CD Pipeline\nAWS CodePipeline: Set up a continuous integration and continuous deployment (CI/CD) pipeline with CodePipeline to automate the build, test, and deployment phases. Caching\nAmazon ElastiCache: Use ElastiCache (Redis or Memcached) to cache frequently accessed data, reducing load on the database and speeding up response times. Cost Management\nAWS Cost Explorer: Use Cost Explorer to monitor and manage your AWS spending. AWS Budgets: Set budgets and receive alerts when you approach your budget limits. By leveraging these AWS tools, you can create a scalable, reliable, and fast website for your small shop, ensuring a smooth experience for users regardless of traffic volume.\nImagine you\u0026rsquo;re like a tech detective investigating why a website\u0026rsquo;s database is slow for a busy online shop. How would you use AWS tools to find out what\u0026rsquo;s causing the problem and make the database super fast again? Investigating and resolving database performance issues for a busy online shop on AWS involves a systematic approach. Here’s how you can use AWS tools to diagnose and fix the problem:\nStep-by-Step Investigation: Monitor Database Performance Metrics:\nAmazon CloudWatch: Start by checking the CloudWatch metrics for your database instance (CPU usage, memory usage, disk I/O, and network I/O). Look for any anomalies or spikes during the busy periods. Enable Enhanced Monitoring:\nAmazon RDS Enhanced Monitoring: Enable Enhanced Monitoring for more granular real-time metrics on the operating system and database processes. This provides deeper insights than standard CloudWatch metrics. Analyze Database Logs:\nAmazon RDS Performance Insights: Enable Performance Insights to identify the top queries and processes consuming the most resources. This helps pinpoint slow queries or resource-intensive operations. Database Configuration Check:\nAmazon RDS Console: Review the configuration parameters of your RDS instance. Ensure that parameters like max_connections, innodb_buffer_pool_size (for MySQL), or shared_buffers (for PostgreSQL) are appropriately set for your workload. Query Optimization:\nQuery Analysis Tools: Use tools like EXPLAIN in MySQL or PostgreSQL to analyze slow queries. Look for missing indexes, suboptimal joins, or inefficient query structures. Amazon Aurora Query Plan Management: If using Aurora, leverage Query Plan Management to optimize and stabilize query execution plans. Steps to Make the Database Fast Again: Scaling Resources:\nVertical Scaling: Upgrade to a larger instance type with more CPU, memory, or IOPS if the current instance is under-provisioned. Horizontal Scaling: For read-heavy workloads, consider adding read replicas to offload read traffic from the primary instance. Database Tuning:\nIndex Optimization: Ensure that all frequently accessed tables have appropriate indexes. Use Performance Insights to identify and create missing indexes. Parameter Tuning: Adjust database parameters based on performance metrics and best practices for your database engine. Caching Layer:\nAmazon ElastiCache: Implement caching using Redis or Memcached to reduce the load on the database by caching frequently accessed data. Database Sharding:\nData Partitioning: Consider sharding your database if it’s struggling to handle the load even after vertical scaling. Distribute the data across multiple instances to balance the load. Optimizing Storage:\nProvisioned IOPS: If disk I/O is a bottleneck, switch to Provisioned IOPS (SSD) storage to ensure consistent and high performance. RAID Configuration: For self-managed databases on EC2, configure RAID 0 for higher throughput on EBS volumes. Load Balancing:\nAmazon RDS Proxy: Use RDS Proxy to pool and share database connections, reducing the overhead of opening and closing connections and improving application scalability. Regular Maintenance:\nAutomated Backups and Maintenance: Ensure that automated backups and maintenance tasks are scheduled during off-peak hours to avoid performance degradation during busy periods. Continuous Monitoring and Improvement: Set Up Alerts:\nAmazon CloudWatch Alarms: Create alarms for key performance metrics to get notified of potential issues before they become critical. Regular Audits:\nPerform regular performance audits and adjustments based on changing workload patterns and application growth. Stay Updated:\nKeep your database engine and application dependencies up-to-date with the latest patches and performance improvements. By systematically monitoring, analyzing, and optimizing your database using these AWS tools and best practices, you can ensure that your online shop’s database remains fast and responsive even during peak traffic periods.\nAs the hero for a startup, your mission is to control their AWS costs while still allowing them to grow. How would you do this, making sure they can expand without spending too much money or slowing down their operations? To control AWS costs for a startup while allowing for growth and maintaining performance, follow these strategies:\nRight-Sizing and Cost Optimization:\nRight-Size Instances: Use AWS Cost Explorer to identify underutilized instances. Scale down instances that are consistently underutilized and right-size them based on actual usage patterns. Reserved Instances and Savings Plans: For steady-state workloads, purchase Reserved Instances (RIs) or Savings Plans to save up to 75% compared to On-Demand pricing. Spot Instances: Use Spot Instances for non-critical and flexible workloads. Spot Instances can be significantly cheaper than On-Demand Instances. Auto Scaling: Implement Auto Scaling to dynamically adjust the number of instances based on demand, ensuring you only pay for what you use. Efficient Use of Storage:\nS3 Storage Classes: Use appropriate Amazon S3 storage classes for different types of data. For example, use S3 Standard for frequently accessed data and S3 Infrequent Access or S3 Glacier for archival data. Lifecycle Policies: Set up lifecycle policies to automatically transition objects between different storage classes or delete them after a certain period. Elastic Block Store (EBS) Optimization: Regularly review and delete unused EBS volumes. Use EBS snapshots for backup instead of keeping inactive volumes. Cost-Effective Database Management:\nRight-Sizing Databases: Regularly review your RDS instances for underutilization and resize them accordingly. Consider using Aurora Serverless for variable workloads. Database Engine Choice: Choose cost-effective database engines such as Amazon Aurora, which can provide significant cost savings over traditional RDS instances. Read Replicas and Cache: Use read replicas for read-heavy workloads and Amazon ElastiCache to offload read traffic and reduce the load on your primary database. Serverless Architectures:\nAWS Lambda: Leverage AWS Lambda for event-driven and short-duration tasks. Lambda’s pay-per-use model can be highly cost-effective for certain workloads. API Gateway: Use API Gateway in combination with Lambda to build scalable and cost-effective APIs. Monitoring and Alerts:\nCost Explorer and Budgets: Regularly use AWS Cost Explorer to analyze spending patterns. Set up AWS Budgets to get alerts when costs exceed predefined thresholds. CloudWatch Alarms: Set up CloudWatch Alarms for monitoring key performance and cost metrics to detect and address cost anomalies promptly. Optimize Networking Costs:\nData Transfer: Minimize data transfer costs by keeping data within the same AWS region and using VPC endpoints for private communication between services. Content Delivery: Use Amazon CloudFront to cache content at edge locations, reducing data transfer costs and improving performance for end-users. Regular Cost Reviews and Audits:\nMonthly Cost Reviews: Conduct regular cost reviews to identify areas of high spending and opportunities for optimization. Cost Allocation Tags: Implement cost allocation tags to track and manage costs by project, department, or environment. Leverage Free Tier and Credits:\nAWS Free Tier: Utilize the AWS Free Tier for eligible services to keep initial costs low. AWS Credits: Apply for AWS Activate Credits available to startups for additional cost savings. By implementing these strategies, you can control AWS costs effectively while ensuring your startup can grow and scale without compromising on performance.\nA nonprofit organization needs help migrating their data to AWS. What steps would you take to ensure a smooth transition, considering their limited budget and technical expertise? Migrating a nonprofit organization’s data to AWS with a limited budget and technical expertise requires careful planning and execution. Here are the steps to ensure a smooth transition:\nAssessment and Planning\nRequirements Gathering: Understand the organization’s current infrastructure, data types, and volume. Identify critical applications, dependencies, and peak usage times. Cost Estimation: Use the AWS Pricing Calculator to estimate costs based on the identified requirements. Consider the AWS Free Tier and AWS Nonprofit Credits to offset initial costs. Migration Strategy: Choose an appropriate migration strategy: lift-and-shift, re-platform, or re-architect. Prioritize data and applications for migration based on complexity and importance. Preparation\nTraining and Support: Provide basic AWS training for the organization’s staff using AWS Training and Certification resources. Consider engaging an AWS Partner with experience in nonprofit migrations for additional support. Infrastructure Setup: Set up the initial AWS infrastructure, including VPC, subnets, security groups, and IAM roles. Implement a cost management plan with AWS Budgets and Cost Explorer. Data Migration\nChoose Data Migration Tools: Use AWS Data Migration Service (DMS) for databases. For large-scale data transfers, consider AWS Snowball or AWS Snowcone. Use AWS S3 Transfer Acceleration for faster internet-based transfers. Initial Data Transfer: Perform an initial bulk data transfer to AWS S3 or the relevant AWS service. Use AWS Storage Gateway for hybrid cloud storage during the transition. Application Migration\nLift-and-Shift: For simple lift-and-shift, use AWS Application Migration Service to replicate applications to AWS. Test applications in the new environment to ensure functionality. Re-platforming and Re-architecting: For applications that need modification, use AWS Elastic Beanstalk or AWS Lambda for deployment. Implement any necessary changes to make applications cloud-native. Security and Compliance\nData Security: Ensure data is encrypted in transit and at rest using AWS KMS. Implement AWS WAF and AWS Shield for additional security layers. Compliance: Ensure compliance with relevant regulations and standards using AWS Artifact to access compliance reports and agreements. Testing and Validation\nFunctional Testing: Conduct thorough testing of applications and data integrity after migration. Validate that all services and applications are working as expected in the AWS environment. Performance Testing: Perform load testing to ensure the infrastructure can handle the expected traffic. Optimize resource allocation based on testing results. Cutover and Go-Live\nFinal Data Sync: Perform a final data synchronization to ensure all recent changes are captured. Switch DNS settings to point to the new AWS-hosted environment using Amazon Route 53. Monitoring and Support: Set up Amazon CloudWatch for monitoring infrastructure and application performance. Provide ongoing support and troubleshooting assistance post-migration. Cost Management and Optimization\nRegular Reviews: Conduct regular cost reviews using AWS Cost Explorer and adjust resources as needed. Implement auto-scaling and use AWS Trusted Advisor for cost optimization recommendations. Leverage AWS Credits: Apply for additional AWS Nonprofit Credits to help manage ongoing costs. By following these steps, the nonprofit organization can achieve a smooth transition to AWS, ensuring their data is securely migrated, costs are managed effectively, and their technical expertise is augmented through training and support.\nA startup wants to store their customer data securely on AWS. How would you recommend they do this, considering both cost and security? To store customer data securely on AWS while considering both cost and security, follow these steps:\nData Storage Solutions\nAmazon S3 (Simple Storage Service): Standard Storage: Use Amazon S3 for general-purpose storage. It’s cost-effective, scalable, and durable. S3 Intelligent-Tiering: Automatically moves data to the most cost-effective storage tier based on access patterns. S3 Glacier: For archival storage, use S3 Glacier, which offers lower costs for infrequently accessed data. Amazon RDS (Relational Database Service): Use RDS for structured data. Choose the appropriate database engine (MySQL, PostgreSQL, etc.) based on requirements. For cost savings, use RDS Reserved Instances or RDS Aurora for high performance at a lower cost. Amazon DynamoDB: Use DynamoDB for NoSQL data. It’s fully managed, scales automatically, and offers on-demand pricing. Security Measures\nData Encryption: At Rest: Enable server-side encryption for S3 buckets using S3-Managed Keys (SSE-S3), AWS Key Management Service (SSE-KMS), or customer-provided keys (SSE-C). In Transit: Use SSL/TLS to encrypt data in transit between the client and AWS services. Access Control: IAM Policies: Use AWS Identity and Access Management (IAM) to define granular permissions for users and services. Follow the principle of least privilege. Bucket Policies: Apply S3 bucket policies and Access Control Lists (ACLs) to control access to S3 objects. Database Authentication: Use IAM database authentication for RDS and DynamoDB, and enable multi-factor authentication (MFA) for added security. Monitoring and Auditing: AWS CloudTrail: Enable CloudTrail to log all API calls for auditing and compliance purposes. Amazon CloudWatch: Set up CloudWatch for monitoring and alerting on critical events and thresholds. AWS Config: Use AWS Config to track configuration changes and ensure compliance with security policies. Data Backup and Recovery: S3 Versioning: Enable versioning in S3 to keep multiple versions of objects and protect against accidental deletions. Automated Backups: Set up automated backups for RDS and enable point-in-time recovery. DynamoDB Backups: Use DynamoDB on-demand backups and point-in-time recovery for continuous data protection. Cost Optimization\nStorage Classes: Use S3 Intelligent-Tiering to automatically move data to the most cost-effective storage class based on access patterns. For infrequently accessed data, use S3 Standard-IA or S3 One Zone-IA. Reserved Instances and Savings Plans: Purchase Reserved Instances for RDS to save costs on predictable workloads. Consider AWS Savings Plans for a flexible pricing model that provides significant savings compared to On-Demand pricing. Lifecycle Policies: Implement S3 lifecycle policies to automatically transition data between different storage classes and delete objects after a specified period. Compliance and Governance\nAWS Artifact: Use AWS Artifact to access AWS compliance reports and agreements, ensuring the startup meets industry standards and regulations. AWS Security Hub: Enable AWS Security Hub to get a comprehensive view of security alerts and compliance status across AWS accounts. Implementation Plan Setup Storage Solutions: Create S3 buckets with appropriate access controls and enable server-side encryption. Set up RDS or DynamoDB instances with encryption and automated backups. Configure Security Measures: Apply IAM policies, bucket policies, and security groups. Enable CloudTrail, CloudWatch, and AWS Config for monitoring and auditing. Optimize Costs: Implement S3 lifecycle policies and use Reserved Instances or Savings Plans. Monitor costs regularly using AWS Cost Explorer and set up AWS Budgets. Regular Reviews and Updates: Regularly review and update security policies and configurations. Stay informed about new AWS services and features that could enhance security and reduce costs. By following these steps, the startup can securely store customer data on AWS while optimizing costs and ensuring compliance with industry standards.\n","permalink":"https://blogs.rameskum.com/posts/aws-question/","summary":"\u003ch2 id=\"aws-use-case-questions\"\u003eAWS Use Case Questions\u003c/h2\u003e\n\u003ch3 id=\"youre-setting-up-a-website-for-a-small-shop-using-aws-how-would-you-choose-the-right-aws-tools-to-make-sure-the-website-stays-fast-and-reliable-whether-there-are-only-a-few-visitors-or-a-lot-of-people-shopping-at-once-during-a-big-sale\"\u003eYou\u0026rsquo;re setting up a website for a small shop using AWS. How would you choose the right AWS tools to make sure the website stays fast and reliable, whether there are only a few visitors or a lot of people shopping at once during a big sale?\u003c/h3\u003e\n\u003cp\u003eTo set up a website for a small shop using AWS, ensuring it stays fast and reliable regardless of traffic fluctuations, follow these steps:\u003c/p\u003e","title":"Aws Questions"},{"content":" Docker Installation Storage Storage Drivers Running Images Logging Driver Docker Swarm Namespaces Control Group Image Dockerfile Multi-Stage Build Managing Image Flattening an Image Docker Installation We have two versions of Docker.\nDocker Community Version (Docker CE) Docker Enterprise (Docker EE) Instalation Guide\nUbuntu Debian Storage By default all files created inside a container are stored on a writable container layer. This means that:\nThe data doesn\u0026rsquo;t persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it. A container\u0026rsquo;s writable layer is tightly coupled to the host machine where the container is running. You can\u0026rsquo;t easily move the data somewhere else. Writing into a container\u0026rsquo;s writable layer requires a storage driver to manage the filesystem. The storage driver provides a union filesystem, using the Linux kernel. This extra abstraction reduces performance as compared to using data volumes, which write directly to the host filesystem. Storage Drivers The storage driver controls how images and containers are stored and managed on your Docker host.\nThere are multiple storage drivers supported by Docker:\noverlay2 fuse-overlays btrfs and zfs vfs Running Images docker run [OPTIONS] IMAGE[:TAG] [COMMAND] [ARG..] Some commonly-used options:\n-d: Run the container in detached mode. --name: To give a descriptive name. --restart: Specify when the container should restart. no (Default): Never restart. on-failure: Only if the container fails. always: always restart the container. unless-stopped: unless manually stopped. -p \u0026lt;host port\u0026gt;:\u0026lt;container port\u0026gt;: Expose port --rm: Automatically remove the container when it exits. Cannot be used with --restart --memory: Hard limit on memory usage. --memory-reservation: A soft limit on memory usage. It will activate when the host is running low on memory. docker run hello-world docker run nginx:1.15.11 # specifying the tag docker run busybox echo hello world! # sending command to run on container docker run -d nginx:1.15.11 # run the container in detached mode docker run -d --name nginx --restart always nginx:1.15.11 docker run -d --name nginx --restart unless-stopped -p 8080:80 --memory 500M --memory-reservation 256M nginx:1.15.11 Logging Driver Docker includes multiple logging mechanisms to help you get information from running containers and services. These mechanisms are called logging drivers. Each Docker daemon has a default logging driver, which each container uses unless you configure it to use a different logging driver, or log driver for short.\nOverriding log driver for a Container\ndocker run -rm --log-driver syslog nginx docker run -rm --log-driver json-file --log-opt max-size=50m nginx Docker Swarm Docker Swarm is a Docker-native clustering system that allows you to manage a group of machines as a single, virtual host. It enables you to run containers across multiple hosts and automates the deployment of containers across a cluster.\nWith Docker Swarm, you can easily scale your services, run the same container on multiple hosts, and manage the underlying infrastructure. It provides a simple command-line interface and a REST API for managing your services.\nTo create swarm manager docker swarm init To join a swarm cluster docker swarm join worker # to get join command with token To get the node details docker node ls Namespaces Docker uses namespaces to provide the isolated workspace called the container. The Docker engine uses namespaces such as the following on Linux:\nThe pid namespace: Process isolation The net namespace: Managing network interfaces The ipc namespace: Manage access to IPC resources The mnt namespace: Managing filesystem mounts The uts namespace: Isolating kernel and version identifiers. The user namespace: It allows a container process to run as root inside the container while mapping to a different unprivileged user on the host. Control Group A cgroup limits an application to a specific set of resources. Control groups allow Docker Engine to share available hardware resources with containers and, optionally, limits and constraints.\nImage An image is a read-only template with instructions for creating a Docker container. It contains the filesystem changes and configurations made when building a Docker image. An image typically contains the application and its dependencies. Images are created from Dockerfiles or can be pulled from a Docker registry. Once an image is created, it can be used to run multiple containers.\nIn Docker, an image is built using a Dockerfile, which is a text file that contains the instructions to build an image. The Dockerfile specifies the base image, installs dependencies, copies files, and sets environment variables.\nTo build an image, you can use the docker build command. Provide the path to the Dockerfile and, optionally, a tag for the image. The tag is used to identify the image later.\nImages are built in layers.\nDockerfile It is used to create an image. A sample docker file example:\n# Simple nginx image FROM ubuntu:bionic ENV NGINX_VERSION 1.14.* RUN apt-get update \u0026amp;\u0026amp; apt-get install -y curl RUN apt-get update \u0026amp;\u0026amp; apt-get install -y nginx=$NGINX_VERSION WORKDIR /var/www/html # WORKDIR www # its a relative path since there is no / ADD index.html ./ # COPY command can also be used, but add has some extra features EXPOSE 80 CMD [\u0026#34;nginx\u0026#34;, \u0026#34;-g\u0026#34;, \u0026#34;daemon off;\u0026#34;] STOPSIGNAL SIGTERM HEALTHCHECK CMD curl localhost:80 Make the docker image faster\nimage should be ephemeral, easy to start, stop and restart Put things that are less likely to change on lower-level layers Don\u0026rsquo;t create unnecessary layers Avoid including any unnecessary files, packages, etc. in the image. Multi-Stage Build Multi-stage builds have more than one FROM directive in the Dockerfile, with each FROM directive starting a new stage. Simple example:\n# name the first compiler stage to be referenced in later layer FROM golang:1.12.4 AS compiler WORKDIR /helloworld COPY helloworld.go. RUN GOOS=linux go build -a -installsuffix cgo -o helloworld . # start a new layer stage FROM alpine:3.9.3 WORKDIR /root COPY --from=compiler /helloworld/helloworld . CMD L\u0026#34; -/helloworld\u0026#34;] Managing Image # to pull the image docker image pull \u0026lt;image name\u0026gt; # to list all the images docker image ls # list all images docker image ls -a # inspect an image docker image inspect nginx docker image inspect nginx --format \u0026#34;{{.Architecture}} {{.Os}}\u0026#34; # delete the image docker image rm nginx docker rmi nginx # delete all dangling images docker image prune Flattening an Image Run a container from the image Export the container to an archive: docker export Import the archive as a new image using docker import # run the image docker run --name abc -d nginx # export the container docker export abc \u0026gt; abc.tar # import the tar file cat abc.tar | docker import - name:latest ","permalink":"https://blogs.rameskum.com/posts/docker/","summary":"\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#docker\"\u003eDocker\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#installation\"\u003eInstallation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#storage\"\u003eStorage\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#storage-drivers\"\u003eStorage Drivers\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#running-images\"\u003eRunning Images\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#logging-driver\"\u003eLogging Driver\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#docker-swarm\"\u003eDocker Swarm\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#namespaces\"\u003eNamespaces\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#control-group\"\u003eControl Group\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#image\"\u003eImage\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#dockerfile\"\u003eDockerfile\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#multi-stage-build\"\u003eMulti-Stage Build\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#managing-image\"\u003eManaging Image\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#flattening-an-image\"\u003eFlattening an Image\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"docker\"\u003eDocker\u003c/h2\u003e\n\u003ch3 id=\"installation\"\u003eInstallation\u003c/h3\u003e\n\u003cp\u003eWe have two versions of Docker.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eDocker Community Version (Docker CE)\u003c/li\u003e\n\u003cli\u003eDocker Enterprise (Docker EE)\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cstrong\u003eInstalation Guide\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://docs.docker.com/desktop/install/ubuntu/\"\u003eUbuntu\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://docs.docker.com/desktop/install/debian/\"\u003eDebian\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"storage\"\u003eStorage\u003c/h3\u003e\n\u003cp\u003eBy default all files created inside a container are stored on a writable container layer. This means that:\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eThe data doesn\u0026rsquo;t persist when that container no longer exists, and it can be difficult to get the data out of the container if another process needs it.\u003c/li\u003e\n\u003cli\u003eA container\u0026rsquo;s writable layer is tightly coupled to the host machine where the container is running. You can\u0026rsquo;t easily move the data somewhere else.\u003c/li\u003e\n\u003cli\u003eWriting into a container\u0026rsquo;s writable layer requires a storage driver to manage the filesystem. The storage driver provides a union filesystem, using the Linux kernel. This extra abstraction reduces performance as compared to using data volumes, which write directly to the host filesystem.\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch4 id=\"storage-drivers\"\u003eStorage Drivers\u003c/h4\u003e\n\u003cp\u003eThe storage driver controls how images and containers are stored and managed on your Docker host.\u003c/p\u003e","title":"Docker"},{"content":"Core Java Core Java String Concepts HashCode \u0026amp; Equal Methods Immutability OOPS Concepts Abstraction Encapsulation Polymorphism Inheritance String Concepts A String is a sequence of characters.\nHow to create a String object?\nString literal\nString s1=\u0026#34;Welcome\u0026#34;; String s2=\u0026#34;Welcome\u0026#34;;//It doesn\u0026#39;t create a new instance New keyword\nString s=new String(\u0026#34;Welcome\u0026#34;); //creates object and reference variable Examples:-\nString s1 = \u0026#34;Hello\u0026#34;; char ch[] = {\u0026#39;H\u0026#39;, \u0026#39;e\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;l\u0026#39;, \u0026#39;o\u0026#39;}; String s2 = new String(ch); String s3 = \u0026#34;Hello\u0026#34;; String s4 = new String(\u0026#34;Hello\u0026#34;); String s5 = new String(\u0026#34;Hello\u0026#34;); System.out.println(s1 == s2); // false System.out.println(s1 == s3); // true System.out.println(s4 == s5); // false HashCode \u0026amp; Equal Methods Default implementations of equals() and hashcode() methods:\nequals() - will return true when the reference points to the same memory address. hashcode() - calculated based on the memory address. We can override the hashcode() and equals() methods, and the contract says both methods should be overridden in such a way that, if the two objects are equal, then the hash-code should be the same as well.\nImmutability Immutability is a software engineering concept stating that an object can not be modified once created. In Java, objects are mutable by default, meaning they can be changed after creation.\nTo create an immutable class, we must do the following:\nDeclare the class as final so it can\u0026rsquo;t be extended. Make all of the fields private so that direct access is not allowed. Don\u0026rsquo;t provide setter methods for variables. Make all mutable fields final so that a field\u0026rsquo;s value can be assigned only once. Initialize all fields using a constructor method to perform a deep copy. Perform cloning of objects in the getter methods to return a copy rather than returning the actual object reference. public final class ImmutableClassExample { private final int id; private final String name; private final HashMap\u0026lt;String,String\u0026gt; roles; public ImmutableClassExample(int id, String name, HashMap\u0026lt;String,String\u0026gt; roles) { this.id = id; this.name = name; HashMap\u0026lt;String,String\u0026gt; rolesCopy = new HashMap\u0026lt;\u0026gt;(); String key; Iterator\u0026lt;String\u0026gt; it = roles.keySet().iterator(); while (it.hasNext()){ key = it.next(); rolesCopy.put(key, roles.get(key)); } this.roles = rolesCopy; } public int getId() { return id; } public String getName() { return name; } public HashMap\u0026lt;String,String\u0026gt; getRoles() { return (HashMap\u0026lt;String,String\u0026gt;) (roles).clone(); } } OOPS Concepts Abstraction Abstraction is the process of showing only necessary information to the outside world while hiding the implementation details. It simplifies a complex system by hiding its complexities and showing only the necessary information. It is used to reduce complexity by only showing the required information to the outside world and hiding the implementation details.\nEncapsulation Encapsulation is the process of combining related data and functionality into a single unit. This unit is called an object. It allows for abstraction, hiding implementation details, and increased code reusability. Objects can also interact with each other through methods.\nPolymorphism Polymorphism is a concept in object-oriented programming that allows objects of different types to be treated as if they were of the same type. Polymorphism allows us to write code that can work with various kinds of objects without explicitly writing code for each type.\nInheritance Inheritance is a feature of object-oriented programming that allows one class to inherit the attributes and methods of another class. The class that is being inherited is called the superclass, parent class, or base class. The class that inherits is called the subclass, child class, or derived class. Inheritance allows for code reusability and helps to promote modularity and abstraction in a program. It also provides a way to create a hierarchy of classes, which can be used to model real-world concepts.\n","permalink":"https://blogs.rameskum.com/posts/core-java/","summary":"\u003ch2 id=\"core-java\"\u003eCore Java\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#core-java\"\u003eCore Java\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#string-concepts\"\u003eString Concepts\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#hashcode--equal-methods\"\u003eHashCode \u0026amp; Equal Methods\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#immutability\"\u003eImmutability\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#oops-concepts\"\u003eOOPS Concepts\u003c/a\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"#abstraction\"\u003eAbstraction\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#encapsulation\"\u003eEncapsulation\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#polymorphism\"\u003ePolymorphism\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"#inheritance\"\u003eInheritance\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"string-concepts\"\u003eString Concepts\u003c/h3\u003e\n\u003cp\u003eA String is a sequence of characters.\u003c/p\u003e\n\u003cp\u003e\u003cstrong\u003eHow to create a String object?\u003c/strong\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\n\u003cp\u003eString literal\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es1\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Welcome\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es2\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Welcome\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"c1\"\u003e//It doesn\u0026#39;t create a new instance\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003c/li\u003e\n\u003cli\u003e\n\u003cp\u003eNew keyword\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es\u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Welcome\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"c1\"\u003e//creates object and reference variable\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003e\u003cem\u003eExamples:-\u003c/em\u003e\u003c/p\u003e\n\u003cdiv class=\"highlight\"\u003e\u003cpre tabindex=\"0\" class=\"chroma\"\u003e\u003ccode class=\"language-java\" data-lang=\"java\"\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es1\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Hello\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"kt\"\u003echar\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003ech\u003c/span\u003e\u003cspan class=\"o\"\u003e[]\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"p\"\u003e{\u003c/span\u003e\u003cspan class=\"sc\"\u003e\u0026#39;H\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"sc\"\u003e\u0026#39;e\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"sc\"\u003e\u0026#39;l\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"sc\"\u003e\u0026#39;l\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e,\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"sc\"\u003e\u0026#39;o\u0026#39;\u003c/span\u003e\u003cspan class=\"p\"\u003e};\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es2\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003ech\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es3\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Hello\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e;\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es4\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Hello\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es5\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e=\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"k\"\u003enew\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003eString\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"s\"\u003e\u0026#34;Hello\u0026#34;\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003es1\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e==\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es2\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"c1\"\u003e// false\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003es1\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e==\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es3\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"c1\"\u003e// true\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003cspan class=\"line\"\u003e\u003cspan class=\"cl\"\u003e\u003cspan class=\"n\"\u003eSystem\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eout\u003c/span\u003e\u003cspan class=\"p\"\u003e.\u003c/span\u003e\u003cspan class=\"na\"\u003eprintln\u003c/span\u003e\u003cspan class=\"p\"\u003e(\u003c/span\u003e\u003cspan class=\"n\"\u003es4\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"o\"\u003e==\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"n\"\u003es5\u003c/span\u003e\u003cspan class=\"p\"\u003e);\u003c/span\u003e\u003cspan class=\"w\"\u003e \u003c/span\u003e\u003cspan class=\"c1\"\u003e// false\u003c/span\u003e\u003cspan class=\"w\"\u003e\n\u003c/span\u003e\u003c/span\u003e\u003c/span\u003e\u003c/code\u003e\u003c/pre\u003e\u003c/div\u003e\u003ch3 id=\"hashcode--equal-methods\"\u003eHashCode \u0026amp; Equal Methods\u003c/h3\u003e\n\u003cp\u003eDefault implementations of \u003ccode\u003eequals()\u003c/code\u003e and \u003ccode\u003ehashcode()\u003c/code\u003e methods:\u003c/p\u003e","title":"Core Java"},{"content":"Kafka - Distributed Stream Processing System Kafka is an open-source distributed streaming platform. In simpler terms, it\u0026rsquo;s a system that excels at handling large amounts of data that is constantly being generated. This data, often called streaming data, comes from various sources and needs to be processed quickly and efficiently.\nPush Poll Model Kafka Components Kafka Broker Default port 9092\nProducer: Producer produces content Consumer: Consumer consumes the content The producer/Consumer creates a TCP connection to the broker, which is bi-directional. That means both producer and broker can send and receive data from each other.\nTopics: Logical partitions where the consumer writes content to. It is mandatory for the consumer to specify the topic name for the content to be written, where as consumer need to specify from which topic to read from. Every message is assigned a position, and it is fast addressable. The consumer is pooling for more messages, unlike rabbit-mq. What to do if the topic grows large?\nSharding in case of the databases if the table grows large. Kafka borrows the same concept, called partitions. The producer now needs to figure out not only which topic to publish data to but also which partition to publish to. Queue vs. PubSub Queue: Message published once, consumed once. PubSub: Message published once, consumed many times. Kafka asked: How can we do both? Answer: Consumer Group\nConsumer Group Invented to do parallel processing on partitions. Consumer groups remove the awareness of partitions from the consumer.\nTo act like a queue, put all your consumers in one group. To act like a pub/sub, put each consumer in a unique group. We get parallel processing for free. Distributed System Spin up another broker, Kafka then marks Leader and follower.\nIt is possible that one broker be leader for one partition and a follower for another partition.\nBut where is the leader information store?\nMeet ZooKeeper\nExample Spin up Kafka cluster # run zookeeper docker run --name zookeeper -p 2181:2181 -d zookeeper # run kafka broker - change the hostname docker run -p 9092:9092 --name kafka -e KAFKA_ZOOKEEPER_CONNECT=MacBook-Air.local:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://MacBook-Air.local:9092 -e KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR=1 -d confluentinc/cp-kafka Write node js Producer/ Creating a Topic\nconst { Kafka } = require(\u0026#39;kafkajs\u0026#39;); async function run() { try { const kafka = new Kafka({ clientId: \u0026#39;myapp\u0026#39;, brokers: [\u0026#39;MacBook-Air.local:9092\u0026#39;], }); const admin = kafka.admin(); console.log(\u0026#39;connecting...\u0026#39;); await admin.connect(); console.log(\u0026#39;connected.\u0026#39;); // A-M, N-Z await admin.createTopics({ topics: [ { topic: \u0026#39;users\u0026#39;, numPartitions: 2, }, ], }); console.log(\u0026#39;topic created successfully\u0026#39;); await admin.disconnect(); } catch (ex) { console.error(`something went wrong ${ex}`); } finally { process.exit(0); } } run(); Creating a Producer\nconst { Kafka } = require(\u0026#39;kafkajs\u0026#39;); const msg = process.argv[2]; async function run() { try { const kafka = new Kafka({ clientId: \u0026#39;myapp\u0026#39;, brokers: [\u0026#39;MacBook-Air.local:9092\u0026#39;], }); const producer = kafka.producer(); console.log(\u0026#39;connecting...\u0026#39;); await producer.connect(); console.log(\u0026#39;connected.\u0026#39;); // A-M 0, N-Z 1 const partition = msg[0] \u0026lt; \u0026#39;N\u0026#39; ? 0 : 1; const result = await producer.send({ topic: \u0026#39;users\u0026#39;, messages: [ { value: msg, partition: partition, }, ], }); console.log(`send successfully: ${JSON.stringify(result)}`); await producer.disconnect(); } catch (ex) { console.error(`something went wrong ${ex}`); } finally { process.exit(0); } } run(); Creating a Consumer\nconst { Kafka } = require(\u0026#39;kafkajs\u0026#39;); async function run() { try { const kafka = new Kafka({ clientId: \u0026#39;myapp\u0026#39;, brokers: [\u0026#39;MacBook-Air.local:9092\u0026#39;], }); const consumer = kafka.consumer({ groupId: \u0026#39;group-1\u0026#39;, }); console.log(\u0026#39;connecting...\u0026#39;); await consumer.connect(); console.log(\u0026#39;connected.\u0026#39;); await consumer.subscribe({ topic: \u0026#39;users\u0026#39;, fromBeginning: true, }); await consumer.run({ eachMessage: async (result) =\u0026gt; { console.log( `Received message ${result.message.value} on partition ${result.partition}` ); }, }); } catch (ex) { console.error(`something went wrong ${ex}`); } finally { } } run(); Pros \u0026amp; Cons of Kafka Pros Append only Commit log Performance: Reading and Writing is fast. Distributed Long Polling Event driver, Pub sub, and Queue Scaling Parallel Processing Cons Zookeeper Producer-explicit partition can lead to problems It is complex to install, configure, and manage Questions The messages in a partition in Kafka consumed sequentially by multiple consumers? Key Concepts:\nTopic: A category or feed name to which messages are published. Partition: A partition is a division of a topic’s log. Each partition is an ordered, immutable sequence of messages. Consumer Group: A group of consumers that work together to consume a topic. Each consumer in the group is assigned to one or more partitions exclusively. How Consumption Works:\nEach partition is assigned to only one consumer within a consumer group. The consumer reads messages from the assigned partition sequentially. This means that messages are consumed in the order they were produced. Multiple consumers can be part of the same consumer group, and Kafka will distribute the partitions among them. However, within a single partition, the order of messages is maintained and consumed by only one consumer at a time. Example:\nConsider a topic with 3 partitions (P0, P1, P2) and a consumer group with 3 consumers (C1, C2, C3):\nP0 might be assigned to C1. P1 might be assigned to C2. P2 might be assigned to C3. Each consumer will consume messages from their respective partitions sequentially. If you add more consumers than partitions, some consumers will be idle as partitions cannot be split further.\n","permalink":"https://blogs.rameskum.com/posts/kafka/","summary":"\u003ch2 id=\"kafka---distributed-stream-processing-system\"\u003eKafka - Distributed Stream Processing System\u003c/h2\u003e\n\u003cp\u003eKafka is an open-source distributed streaming platform. In simpler terms, it\u0026rsquo;s a system that excels at handling large amounts of data that is constantly being generated. This data, often called streaming data, comes from various sources and needs to be processed quickly and efficiently.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003ePush Poll Model\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"kafka-components\"\u003eKafka Components\u003c/h3\u003e\n\u003ch4 id=\"kafka-broker\"\u003eKafka Broker\u003c/h4\u003e\n\u003cp\u003eDefault port \u003ccode\u003e9092\u003c/code\u003e\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003e\u003cstrong\u003eProducer\u003c/strong\u003e: Producer produces content\u003c/li\u003e\n\u003cli\u003e\u003cstrong\u003eConsumer\u003c/strong\u003e: Consumer consumes the content\u003c/li\u003e\n\u003c/ul\u003e\n\u003cp\u003eThe producer/Consumer creates a TCP connection to the broker, which is bi-directional. That means both producer and broker can send and receive data from each other.\u003c/p\u003e","title":"Apache Kafka Crash Course"},{"content":"AWS Certified Developer - Associate showcases knowledge and understanding of core AWS services, uses, and basic AWS architecture best practices, and proficiency in developing, deploying, and debugging cloud-based applications by using AWS. Preparing for and attaining this certification gives certified individuals more confidence and credibility. Organizations with AWS Certified developers have the assurance of having the right talent to give them a competitive advantage and ensure stakeholder and customer satisfaction.\nTo learn more about the \u0026ldquo;AWS Certified Developer - Associate\u0026rdquo; exam refer https://aws.amazon.com/certification/certified-developer-associate/\nTo check if there is any upcoming changes in any AWS exam refer https://aws.amazon.com/certification/coming-soon/\nExam Overview Level Associate Length 130 minutes to complete the exam. Cost 150 USD Format 65 questions, either multiple choice or multiple responses. Delivery method Pearson VUE testing center or online proctored exam. Passing ~72% **could be changed. Download Exam Guide | Download Sample Questions\nWho should take this exam? AWS Certified Developer - Associate is a great starting point on the AWS Certification path for individuals who may have any of the following:\nExperience working in a developer role with in-depth knowledge of at least one high-level programming language. Experience in AWS technology. Strong on-premises IT experience and understanding of mapping on-premises to the cloud. Experience working in other cloud services. Personal Notes Code DVA-C01 / DVA-C02 (latest), both codes have the same syllabus and only the module distribution has changed. Average Time 2 mins / question Types of Questions Multiple Choice and Multiple Responses Exam Breakdown The whole exam is broken down into domains and sub-domains.\nDeployment ~24% roughly 15-16 Questions Security ~26% roughly 16-17 Questions Development ~32% roughly 20-21 Questions Monitoring and Troubleshooting ~18% roughly 11-12 Questions Recommended Whitepaper Architecting for the Cloud: AWS Best Practices Practicing Continuous Integration and Continuous Delivery on AWS Accelerating Software Delivery with DevOps Introduction to Elastic Beanstalk It is a PaaS that allows you to quickly deploy and manage web apps on AWS without worrying about the underlying infrastructure.\nWhat is Platform as a Service? (PaaS) A platform allowing customers to develop, run, and manage applications without the complexity of building and maintaining the infrastructure typically associated with developing and launching an app.\nPaaS is like renting a pre-built cloud workspace for developing and running applications. It saves time and money and allows for easy scaling.\nIt is not recommended for \u0026ldquo;Production\u0026rdquo; applications. Talking about enterprise, large companies.\nPowered by a CloudFormation template setup for you:\nElastic Load Balancer Autoscaling Groups RDS Database EC2 Instance pre-configured (or custom) platforms Monitoring (CloudWatch, SNS) In-place and Blue/Green deployment methodologies Security (Rotates Passwords) Can run Dockerize environments Supported Languages Ruby Python PHP Tomcat NodeJS Web vs. Worker Environment Web Environment Has two variants Load Balanced Environment Uses ASG and set to scale Use an ELB Designed to Scale Variable cost associated based on load Single-Instance Env Still uses an AGS, but Desired Capacity is set to 1 to ensure the server is always running. No ELB to save on cost. The Public IP Address has to be used to route traffic to the server. Creates an ASG (Auto Scaling Group) Creates an ELB (Elastic Load Balance) - optional Worker Environment For backend jobs Creates and ASG Creates and SQS Queue Installs the SQSD daemon on the EC2 Instances Create CloudWatch Alarm to dynamically scale instances based on health. Deployment Policies These are the deployment policies available with the Elastic Beanstalk\nDeployment Policy Load Balanced Env Single Instance Env All at Once 👍 👍 Rolling 👍 ❌ Rolling with additional batch 👍 ❌ Immutable 👍 👍 All at Once Deploy the new app version to all instances at the same time. Takes all instances out of service during the deployment process. Servers become available again The fastest but also the most dangerous deployment method. In case of Failure, you need to roll back the changes by re-deploying the original version again to all instances.\nRolling Deploys the new app version to a batch of instances at a time. Takes batch instances out of service while the deployment processes. Reattaches updated instances. Goes onto the next batch, taking them out of service. Reattaches those instances (rinse and repeat) In Case of Failure, You need to perform an additional rolling update in order to roll back the changes.\nRolling with Additional Batch Rolling with additional batch ensure our capacity is never reduced. This is important for applications where a reduction in capacity could cause availability issues for users.\nLaunch a new instance that will be used to replace a batch. Deploy update app version to new batch. Attach the new batch and terminate the existing batch. In case of failure, you need to perform an additional rolling update to roll back the changes.\nImmutable Create a new ASG with EC2 instances. Deploy the updated version of the app on the new EC2 instances. Point the ELB to the new ASG and delete the old ASG, which will terminate the old EC2 instances. The safest way to deploy for critical applications. In case of failure just terminate the new instances since the existing instances still remain.\nEB - Deployment Methods Method Impact of failed deployment Deploy time No downtime No DNS change Rollback progress code deployed to Instances All at once Downtime ⏰ ❌ 👍 Manual Existing Rolling Single batch out of service; any successful batches before failure running new application version ⏰⏰ * 👍 👍 Manual Existing Rolling with additional batch Minimal if first batch fails; otherwise, similar to Rolling ⏰⏰⏰ * 👍 👍 Manual New and Existing Immutable Minimal ⏰⏰⏰⏰ 👍 👍 Terminate New New Blue/Green Minimal ⏰⏰⏰⏰ 👍 ❌ Swap URL New * Time may vary\nEB - In Place vs. Blue/Green Deployment Elastic Beanstalk, by default, performs in-place updates.\n👋🏼 In-Place and Blue/Green Deployment are not definitive in definition and the context can change the scope of what they mean.\n","permalink":"https://blogs.rameskum.com/posts/aws-developer-certification/","summary":"\u003cp\u003eAWS Certified Developer - Associate showcases knowledge and understanding of core AWS services, uses, and basic AWS architecture best practices, and proficiency in developing, deploying, and debugging cloud-based applications by using AWS. Preparing for and attaining this certification gives certified individuals more confidence and credibility. Organizations with AWS Certified developers have the assurance of having the right talent to give them a competitive advantage and ensure stakeholder and customer satisfaction.\u003c/p\u003e","title":"Aws Developer Certification"},{"content":"The cloud computing industry is booming, and Amazon Web Services (AWS) is at the forefront. With businesses increasingly migrating their operations to the cloud, the demand for skilled AWS professionals is high. If you\u0026rsquo;re looking to upskill yourself and advance your career in cloud computing, then achieving an AWS Certification is a strategic move.\nWhat is AWS Certification? AWS Certification validates your knowledge and expertise in using the AWS cloud platform. It\u0026rsquo;s a way to showcase your abilities to potential employers and demonstrate your understanding of cloud concepts, security, architecture, and various AWS services.\nBenefits of AWS Certification There are numerous advantages to becoming AWS certified:\nIncreased Earning Potential: Studies show that AWS certified professionals can command higher salaries compared to their non-certified counterparts. Career Advancement: An AWS certification can open doors to new job opportunities and promotions within the cloud computing field. Validation of Skills: The certification process verifies your knowledge and practical abilities, giving employers confidence in your expertise. Enhanced Credibility: AWS certifications are recognized worldwide, making you a more attractive candidate in the job market. Staying Relevant: The cloud landscape is constantly evolving, and AWS certifications ensure you possess the latest knowledge and best practices. Choosing the Right AWS Certification AWS offers a variety of certifications catering to different roles and experience levels. Here\u0026rsquo;s a quick breakdown of some popular options:\nAWS Certified Cloud Practitioner: Ideal for beginners seeking a foundational understanding of the AWS cloud. AWS Certified Solutions Architect - Associate: Validates your ability to design and deploy secure, reliable, and cost-effective cloud solutions on AWS. AWS Certified SysOps Administrator - Associate: Focuses on deployment, management, and operations of AWS services. AWS Certified Developer - Associate: Geared towards developers who build and deploy applications on AWS. Please refer https://aws.amazon.com/certification/ for more details.\nPreparing for Your AWS Certification Exam There are several resources available to help you prepare for your AWS certification exam. Amazon offers a range of training materials, including video courses, practice exams, and whitepapers https://aws.amazon.com/training/. Additionally, numerous online platforms and bootcamps provide comprehensive study guides and practice tests.\nConclusion Earning an AWS certification is a valuable investment in your cloud computing career. It demonstrates your commitment to professional development and positions you for success in the ever-growing cloud industry. So, if you\u0026rsquo;re serious about propelling your cloud career forward, consider pursuing an AWS certification today!\n","permalink":"https://blogs.rameskum.com/posts/aws-certification/","summary":"\u003cp\u003eThe cloud computing industry is booming, and Amazon Web Services (AWS) is at the forefront. With businesses increasingly migrating their operations to the cloud, the demand for skilled AWS professionals is high. If you\u0026rsquo;re looking to upskill yourself and advance your career in cloud computing, then achieving an AWS Certification is a strategic move.\u003c/p\u003e\n\u003ch2 id=\"what-is-aws-certification\"\u003eWhat is AWS Certification?\u003c/h2\u003e\n\u003cp\u003eAWS Certification validates your knowledge and expertise in using the AWS cloud platform. It\u0026rsquo;s a way to showcase your abilities to potential employers and demonstrate your understanding of cloud concepts, security, architecture, and various AWS services.\u003c/p\u003e","title":"Aws Certification"},{"content":"About Me I am a developer by trade with a keen interest in technology. Besides challenging myself, I love learning new technologies.\nFact: I do not fear computers. I fear a lack of them.\nMy Portfolio: rameskum.com Github: /rameskum LinkedIn: in/rameskum Projects Portfolio — My portfolio website. Hello Dog — Random dog images by breed name, written in plain HTML, CSS and Javascript. E-Commerce Admin Dashboard — NextJs based e-commerce admin page. E-Commerce Website — NextJs based e-commerce website. ","permalink":"https://blogs.rameskum.com/about/","summary":"\u003ch2 id=\"about-me\"\u003eAbout Me\u003c/h2\u003e\n\u003cp\u003eI am a developer by trade with a keen interest in technology. Besides challenging myself, I love learning new technologies.\u003c/p\u003e\n\u003cp\u003eFact: I do not fear computers. I fear a lack of them.\u003c/p\u003e\n\u003cul\u003e\n\u003cli\u003eMy Portfolio: \u003ca href=\"https://rameskum.com\"\u003erameskum.com\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eGithub: \u003ca href=\"https://github.com/rameskum\"\u003e/rameskum\u003c/a\u003e\u003c/li\u003e\n\u003cli\u003eLinkedIn: \u003ca href=\"https://www.linkedin.com/in/rameskum/\"\u003ein/rameskum\u003c/a\u003e\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch2 id=\"projects\"\u003eProjects\u003c/h2\u003e\n\u003cul\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/rameskum/portfolio.git\"\u003ePortfolio\u003c/a\u003e — My portfolio website.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://rameskum.github.io/hello-dog/\"\u003eHello Dog\u003c/a\u003e — Random dog images by breed name, written in plain HTML, CSS and Javascript.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/rameskum/ecommerce-admin\"\u003eE-Commerce Admin Dashboard\u003c/a\u003e — NextJs based e-commerce admin page.\u003c/li\u003e\n\u003cli\u003e\u003ca href=\"https://github.com/rameskum/ecommerce-store\"\u003eE-Commerce Website\u003c/a\u003e — NextJs based e-commerce website.\u003c/li\u003e\n\u003c/ul\u003e","title":"About me"},{"content":"Undirected Graphs Some problems Path Shortest path Cycle Ehler tour: A cycle that uses each edge excatly once. Hamilton tour: A cycle that uses each vertex exactly once classical NP-complete problem. Connectivity MST: Biconnectivity: A vertex whose removal disconnects the graph Planarity Graph isomorphism: Are two graphs identical? No one knows so far. A lonstanding open problem Representations Real-world graphs tend to be sparse (huge number of vertices, small average vertex degree).\nSet-of-edges representation unefficient Adjacency-matrix representation space cost is prohibitive Adjacency-list array representation GOOD Adjacency-list Data structure Space usage proportional to V + E Constant time to add an edge Time proportional to the degree of v to iterate through vertices adjacent to v Depth-first Search (DFS) Typical applications:\nFind all vertices connected to a given source vertex Find a path between two vertices Algorithm:\nUse recursion (a function-call stack) or an explicit stack. Mark each visited vertex (and keep track of edge taken to visit it) Return (retrace steps) when no unvisited options public class DepthFirstPaths{ private blloean[] marked; private int[] edgeTO; private int s; public DepthFirstPaths(Graph G, int s) { // ... dfs(G, s); } private void dfs(Graph Gm int v) { marked[v] = true; for (int w : G.adj(v)) if (!marked[v]) { dfs(G, w) edgeTo[w] = v; } } } Propositions:\nDFS marks all vertices connected to s in time proportional to the sum of their degrees. After DFS, can find vertices connected to s in constant time and can find a path to s in time proportional to its length. Breadth-first Search (BFS) Typical applications:\nshortest path Algorithm:\nPut s onto a queue, and mark s as visited Take the next vertex v from the queue and mark it Put onto the queue all unmarked vertices that are adjacent to v public class BreadthFirstPaths { private boolean[] marked; private int[] edgeTo; // ... private void bfs(Graph G, int s) { Queue\u0026lt;Integer\u0026gt; q = new Queue\u0026lt;\u0026gt;(); q.enqueue(s); marked[s] = ture; while (!q.isEmpty()) { int v = q.dequeue(); for (int w: G.adj(v)) { if (!marked[w]) { q.enqueue(w); marked[w] = true; edgeTo[w] = v; } } } } } Proposition:\nBFS computes shortest paths (fewest number of edges) from s to all other vertices in a graph in time proportional to E + V Applications of DFS Connected components The goal is to preprocess graph to answer queries of the form is v connected to w? in constant time.\nThe relation is connected to is an equivalence relation:\nReflexive: v is connected to v Symmetric: if v is connected to w, then w is connected to v Transitive: if v connected to w and w connected to x, then v connected to x public class CC { private boolean[] marked; private int[] id; private int count; public CC(Graph G) { marked = new boolean[G.V()]; id = new int[G.V()]; for (int v = 0, v \u0026lt; G.V(); v++) { if (!marked[v]) { dfs(G, v); count++; } } } // ... private void dfs(Graph G, int v) { marked[v] = true; id[v] = count; for (int w : G.adj(v)) { if (!marked[w]) { dfs(G, w) } } } } Cycle detection Problem: Is a given graph acylic?\nTODO\nTwo-colorability Problem: Is the graph bipartite?\nTODO\nSymbol graphs TODO\nDegrees of separation TODO\nDirected Graphs A directed graph (or digraph) is a set of vertices and a collection of directed edges. Each directed edge connects an ordered pair of vertices.\noutdegree: the number of edges going from it indegree: the number fo edges going into it directed path: a sequence of vertices in which there is a (directed) edge pointing from each vertex in the sequence to its successor in the sequence directed cycle simple cycle: a cycle with no repeated edges or vertices Representations Again, use adjacency-lists representation\nBased on iterating over vertices pointing from v Real-world digraphs tend to be sparse public class Digraph { private final int V; private final Bag\u0026lt;Integer\u0026gt;[] adj; public Digraph(int V) { this.V = V; adj = (Bag\u0026lt;Integer\u0026gt;[]) new Bag[V]; for (int v = 0; v \u0026lt; V; v++) { adj[v] = new Bag\u0026lt;Integer\u0026gt;[]; } } public void addEdge(int v, int w) { adj[v].add(w); } public Iterable\u0026lt;Integer\u0026gt; adj(int v) { return adj[v]; } } Digraph search Reachabiliity problem: Find all vertices reachable from s along a directed path.\nWe can use the same dfs method as for undirected graphs.\nEvery undirected graph is a digraph with edges in both directions. DFS is a digraph algorithm, Reachability applications:\nprogram control-flow analysis Dead-code elimination infinite-loop detection mark-sweep garbage collector Other DFS problems:\nPath findind Topological sort Directed cycle detection \u0026hellip; BFS problems:\nshortest path multiple-source shortest paths web crawler application Topological Sort Topological sort: Given a digraph, put the vertices in order such that all its directed edges point from a vertix earlier in the order to a vertex later in the order (or report impossible).\nA digraph has a topological order if and only if it is a directed acyclic graph (DAG). Topological sort redraws DAG so all edges poitn upwards.\nuse DFS again. It can be proved that reverse postorder of a DAG is a topological order. (check P578 for the definition of Preorder/Postorder)\npublic class DepthFirstOrder { private boolean[] marked; private Stack\u0026lt;Integer\u0026gt; reversePost; publiv DepthFirstOrder(Digraph G) { reversePost = new Stack\u0026lt;Integer\u0026gt;(); marked = new boolean[G.V()]; for (int v = 0; v \u0026lt; G.V(); v++) { if (!marked[v]) dfs(G, v); } } private void dfs(Digrapg G, int v) { marked[v] = true; for (int w : G.adj(v)) { if (!marked[w]) dfs(G, w) } reversePost.push(v); } } Directed cycle detection To find out if a given digraph is a DAG, we can try to find a directec cycle in the digraph. Use DFS and a stack to track the cycle.\n// TODO Some very typical applications of directed cycle detection and topological sort: (A directed cycle means the problem is infeasible)\njob schedule course scuedule inheritance spreadsheet vertex: cell edge: formula symbolic links Strong components Vertices v and w are strongly connected if there is both a directed path from v to w and a directed path from w to v. Strong connectivity is an equvicalence relation.\nKosaraju-Sharir Algorithm Kosaraju-Sharir is easy to implement but difficutl to understand. It runs DFS twice:\nGiven a digraph G, run DFS to compute the topological order of its reverse $G^R$ Run DFS on G in the order given by first DFS TODO: ADD Proof\nhttps://algs4.cs.princeton.edu/code/edu/princeton/cs/algs4/KosarajuSharirSCC.java.html\npublic class KosarajuSharirSCC { private boolean[] marked; // marked[v] = has vertex v been visited? private int[] id; // id[v] = id of strong component containing v private int count; // number of strongly-connected components /** * Computes the strong components of the digraph {@code G}. * @param G the digraph */ public KosarajuSharirSCC(Digraph G) { // compute reverse postorder of reverse graph DepthFirstOrder dfs = new DepthFirstOrder(G.reverse()); // run DFS on G, using reverse postorder to guide calculation marked = new boolean[G.V()]; id = new int[G.V()]; for (int v : dfs.reversePost()) { if (!marked[v]) { dfs(G, v); count++; } } } // DFS on graph G private void dfs(Digraph G, int v) { marked[v] = true; id[v] = count; for (int w : G.adj(v)) { if (!marked[w]) dfs(G, w); } } // ... } Minimum Spanning Trees An edge-weighted-graph is a graph where we associate weight or costs with each edge. A spanning tree of an undirected edge-weighted graph G is a subgraph T that is both a tree (conneted and acyclic) and spanning (includes all of the vertices). Given an (connected) undirected edge-weighted graph G with V vertices and E edges, the MST of it must have V - 1 edges. If the graph is not connceted, we compute minimum spanning forest (MST of each component).\nA cut in a graph is a partition of its vertices into two (nonempty) sets A crossing edge connects a vertex in one set with a vertex in the other. Cut property: Given any cut, the crossing edge of min weight is in the MST. Edge-weight Graph Data Type Edge: https://algs4.cs.princeton.edu/code/edu/princeton/cs/algs4/Edge.java.html\nEdgeWeigthedGraph: https://algs4.cs.princeton.edu/code/edu/princeton/cs/algs4/EdgeWeightedGraph.java.html\nGreedy MST Algorithm: Start with all edges colored gray. Find cut with no blacked crossing edges; color its min-weight edge black. Repeat until V-1 edges are colored black. Implementations 1: Kruskal\u0026rsquo;s algorithm For edges in ascending order of weight:\nAdd next edge to Tree unless doing so would create a cycle. To efficiently solve this problem, use union-find :\nuse a priority queue to maintain all the edges in V union-find data structure: maintain a set for each connected component in T. if v and w are in saome set, then adding v-\u0026gt;w would create a cycle to add v\u0026gt;w to T, merge sets containing v and w. TODO: Add code\nImplementations 2: Prim\u0026rsquo;s algorithm Start with vertex 0 and greedily grow tree T. Add To T the min weight edge with exactly oue endpoint in T. Reapeat unitl V - 1 edges. The key to solve this problem is how do we find the crossing edge of minimal weight efficiently.\nA lazy solution (in time proportional to $ElogE$, fair enough):\nMaintain a PQ of edges with (at least) one endpoint in T Key = edge, priority = weight Delete-min to determine next edge e = v-\u0026gt;w to add to T Disregard if both endpoints v and w are marked (both in T) Otherwise, let w be the unmarked vertex (not in T) add to PQ and edge incident to w (assuming other endpoint not in T) add e to T and mark w TODO: add code\nA eager solution (in time proprotional to $ElogV$, better):\nMaintain a PQ of vertices connected by an edge to T, where priority of v = weight of shortedt edge connecting v to T Delete min vertex v and add its associated edge e = v-\u0026gt;w to T Update PQ by considering all edges e = v-\u0026gt;x incident to v ignore if x is already in T add x to PQ if not alread on it decrease priority of x if v-\u0026gt;x becomes shortest edge connecting x to T This solution uses an indexed priority queue data structure.\nTODO: add code\nShortest Paths Some variants:\nWhich vertices? Single source Source-sink All pairs Edge weights Nonegative weights Euclidean weights Arbitrary weights Cycles? No directed cycles No negative cycles Edge-weighted digraph data strcuture Weighted directed edge: https://algs4.cs.princeton.edu/code/edu/princeton/cs/algs4/DirectedEdge.java.html\nEdge-weighted digraph: https://algs4.cs.princeton.edu/code/edu/princeton/cs/algs4/EdgeWeightedDigraph.java.html\nUse adjacency-lists implementation same as EdgeWeightedGraph\nGeneric Single-source Shortest paths Our goal is to find the shortest path from s to every other vertex. As a result, what we find will be the shortest-paths tree (SPT) for source s.\nRelax edge e = v-\u0026gt;w distTo[v] is length of shortest known path from s to v distTo[w] is length of shortest known path from s to w esgeTo[w] is last edge on shortest known pathh from s to w if e = v-\u0026gt;w gives shorter path to w through v, update both distTo[w] and edgeTo[w] private void relax(DirectedEdge e) { int v = e.from(), w = e.to(); if (distTo[w] \u0026gt; distTo[v] + e.weight()) { distTo[w] = distTo[v] + e.weight(); edgeTo[w] = e; } } Optimality conditions Given an edge-weighted digraph G, distTo[] are the shortest path distances from s iff:\ndistTo[s] = 0 For each vertex v, distTo[v] is the length of some path from s to v. For each edge e = v-\u0026gt;w, distTo[w] \u0026lt;= distTo[v] + e.weight() Generic algorithm Generic algorithm (to compute SPT from s) { Initialize distTo[s] = 0 and distTo[v] = $\\infty$ Repeat until optimality conditions are satisfied: - Relax any edge } Efficient implementations:\nNonnegative weights: Dijkstra\u0026rsquo;s algorithm No directed cycles (DAGs): Topological sort algorithm No negative cycles: Bellman-Ford Implement 1: Dijkstra\u0026rsquo;s algorithm When there is no nonnegative weight exists, we can use Dijkstra\u0026rsquo;s algorithm.\nConsider vertices in increasing order of distance from s (non-tree vertex with the lowest distTo[] value) add vertex to tree and relax all edges pointing from that vertex public class DijkstraSP{ // ... public DijkstraSP(EdgeWeightedDigraph G, int s) { edgeTo = new DirectedEdge[G.V()]; distTo = new double[G.V()]; pq = new IndexMinPQ\u0026lt;Double\u0026gt;(G.V()); for (int v = 0; v \u0026lt; G.V(); v++) { distTo[v] = Double.POSITIVE_INFINITY; } distTo[s] = 0; pq.insert(s, 0.0); while(!pq.isEmpty()) { int v= pq.delMin(); for (DirectedEdge e : G.adj(v)) { relax(e); } } } private void relax(DirectedEdge e) { int v = e.from(), w = e.to(); if (distTo[w] \u0026gt; distTo[v] + e.weight()) { distTo[w] = distTo[v] + e.weight(); edgeTo[w] = e; if (pq.contains(w)) pq.decreaseKey(w, distTo[w]); else pq.insert(w, distTo[w]); } } } Compare to Prim\u0026rsquo;s algorithm:\nBoth are computing a graph\u0026rsquo;s spanning tree Prim\u0026rsquo;s algorithm choose closest vertex to tree as next vertex, while Dijkstra\u0026rsquo;s algorithm choose closest vertex to the source Implement 2: Topological sort algorithm When the graph is a DAG, we can consider vertices in topological order and do relaxing.\n// ... Topological topological = new Topological(G); for (int v : topological.order()) { for (DirectedEdge e : G.adj(v)) { relax(e); } } Seam carving: Resize an image without distortion.\nLongest paths:\nFormuate as a shortest paths problem in edge-weighted DAGs Negate all weights Find shortest paths Negate weights in result Allpication: Parallel job scheduling (Critical path method, CPM). Implement 3: Bellman-Ford algorithm A SPT exists iff no negative cycles (a directed cycle whose sum of edge weights is negative).\nWhen we want to find shortest paths with nagative weights, Dijkstra\u0026rsquo;s algorithms doesn\u0026rsquo;t work. We can use Bellman-Ford algorithm as long as there is no negative cycle in the graph. (Bellman-Ford algorithm is a dynamic programming algorithm)\nInitialize distTo[s] = 0 and distTo[v] = $\\infty$ Maintain a queue and repeat until the queue is empty or find a cycle: Pop vertex v from q Relax each edge pointing from v to any vertex w: if distTo[w] can be de decreased, update distTo[w] and add w to the queue // ... public BellmanFordSP(EdgeWeightedDigraph G, int s) { distTo = new double[G.V()]; edgeTo = new DirectedEdge[G.V()]; onQueue = new boolean[G.V()]; for (int v = 0; v \u0026lt; G.V(); v++) distTo[v] = Double.POSITIVE_INFINITY; distTo[s] = 0.0; // Bellman-Ford algorithm queue = new Queue\u0026lt;Integer\u0026gt;(); queue.enqueue(s); onQueue[s] = true; while (!queue.isEmpty() \u0026amp;\u0026amp; !hasNegativeCycle()) { int v = queue.dequeue(); onQueue[v] = false; relax(G, v); } } private void relax(EdgeWeightedDigraph G, int v) { for (DirectedEdge e : G.adj(v)) { int w = e.to(); if (distTo[w] \u0026gt; distTo[v] + e.weight()) { distTo[w] = distTo[v] + e.weight(); edgeTo[w] = e; if (!onQueue[w]) { queue.enqueue(w); onQueue[w] = true; } } if (++cost % G.V() == 0) { findNegativeCycle(); if (hasNegativeCycle()) return; // found a negative cycle } } } Bellman-Ford algorithm can also be used for finding a negative cycle.\nNegative cycle application: arbitrage detection.\n","permalink":"https://blogs.rameskum.com/posts/algorithms-graphs/","summary":"\u003ch2 id=\"undirected-graphs\"\u003eUndirected Graphs\u003c/h2\u003e\n\u003ch3 id=\"some-problems\"\u003eSome problems\u003c/h3\u003e\n\u003cul\u003e\n\u003cli\u003ePath\u003c/li\u003e\n\u003cli\u003eShortest path\u003c/li\u003e\n\u003cli\u003eCycle\u003c/li\u003e\n\u003cli\u003eEhler tour: A cycle that uses each edge excatly once.\u003c/li\u003e\n\u003cli\u003eHamilton tour: A cycle that uses each vertex exactly once\n\u003cul\u003e\n\u003cli\u003eclassical NP-complete problem.\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003cli\u003eConnectivity\u003c/li\u003e\n\u003cli\u003eMST:\u003c/li\u003e\n\u003cli\u003eBiconnectivity: A vertex whose removal disconnects the graph\u003c/li\u003e\n\u003cli\u003ePlanarity\u003c/li\u003e\n\u003cli\u003eGraph isomorphism: Are two graphs identical?\n\u003cul\u003e\n\u003cli\u003eNo one knows so far. A lonstanding open problem\u003c/li\u003e\n\u003c/ul\u003e\n\u003c/li\u003e\n\u003c/ul\u003e\n\u003ch3 id=\"representations\"\u003eRepresentations\u003c/h3\u003e\n\u003cp\u003eReal-world graphs tend to be \u003cstrong\u003esparse\u003c/strong\u003e (huge number of vertices, small average vertex degree).\u003c/p\u003e","title":"Algorithms - Graphs"}]