Skip to main content

Posts

Java ExecutorService and Linux Max Process Limit

Here's a little post about the Java ExecutorService, and a problem I ran into on Ubuntu. Consider this Java code
public static void main(String[] args) throws InterruptedException { int tasksSubmitted = 0; int nTasks = 800; int taskDur = 2000; int nThreads = 400; ExecutorService executor = Executors.newFixedThreadPool(nThreads); try { for (int i = 0; i < nTasks; i++) { executor.submit(() -> { try { Thread.sleep(taskDur); } catch (InterruptedException e) { System.out.println(e); } }); tasksSubmitted++; } } catch (Error | Exception e) { System.out.println(e); } System.out.println("Tasks submitted: " + tasksSubmitted); executor.shutdown(); executor.awaitTermination(Long.MAX_VALUE, TimeUnit.MILLISECONDS); if (tasksSubmitted != nTasks) { System.out.println("Only…
Recent posts

Openers vs Closers

I've read an article in the past about the 2 types of developers: openers and closers. I can't remember the exact link, but this distinction is weighing on my mind lately.

In my mind these are roles, not developer types. A developer might be filling either role, depending on the circumstances.

Openers

The openers are the developers that are on the front line, taking feature requests from managers and customers, and turning them into real applications that can be interacted with. The openers are free to use whatever means necessary to get the job done. They aren't concerned with performance or scalability. Their job is to make the app work.

Closers

Once the app features solidify, the closers are the ones who fix the performance and scalability issues. They care deeply about runtime performance, about efficient algorithms, about profiling tools, memory allocation, all all the deep technical issues. While all developers are typically nerds, the closes are typically the uber-ner…

Relationships Between The DevOps Handbook and Systems Thinking

I've recently finished reading The DevOps Handbook, and Thinking in Systems, and have just started reading The Fifth Discipline. Below are a few of my thoughts after reading them.

The DevOps Handbook is a really great book and I highly recommend anyone participating in software development to give it a read. In the book, the authors describe the Three Ways (aka principles):
FlowFeedback Continual Learning and Experimentation
The first way focuses on getting software through the work stream, the second way focuses on getting feedback, and the third way focuses on organization learning. 
The DevOps Handbook's visualization of the 3 ways reminds me of systems diagrams in Thinking in Systems by Donella Meadows. 

Stuart Rimell has a nice 2 part post on systems thinking in software development, that draws on ideas in Thinking in Systems (Part 1, Part 2)
After reading The DevOps Handbook and Thinking in Systems, I got curious about systems thinking, and how that applies to software de…

Java: throws Exception

I have a pet peeve in Java, and it's methods that use the java.lang.Exception class in the throws clause.

For example

public void doSomething() throws Exception {
}

This forces callers of the doSomething method to either catch the raw Exception, or add it to throws clause of the consuming method. I think that this encourages developers to do the wrong thing. If you're going to use checked exceptions, then callers should only be catching specific exceptions that can be handled. Otherwise, the exception should propagate up the call stack, potentially terminating the thread if no one handles the exception.

Throws clauses are part of the specification of your method. If you're going to use checked exceptions, wouldn't it be nice if the caller could see the explicit list of exceptions to be handled? From a contract perspective, there's no value in specifying that your method can throw any Exception. You're better off removing the throws Exception clause, and just th…

2016 Reading List

As part of my own personal professional development, I publish the books I've read for the year. I'm on GoodReads, and here is my reading challenge.

Looking back, there were a few themes
Performance - I'm staring to get serious about performance analysis of systems. Brendan Gregg and Martin Thompson are influential in my understanding of systems and how to design for performance. It's also clear that what I thought I knew about computers, specifically OS kernels, is severely lacking. Operations - Site Reliability Engineering opened my eyes as to how large companies such as Google support applications in production. Philosophy and religion interest me. The more I think about it, I'm probably an atheist. The scientific method resonates with me, as well as Occam's Razor. I question my Catholic upbringing. The Bill Nye / Ken Ham debate gets you thinking. 
Systems Performance: Enterprise and the Cloud, Prentice Hall, 2013, Brendan GreggExtreme Ownership: How U.S. Navy…

Generating Java Mixed Mode Flame Graphs

Overview I've seen Brendan Gregg's talk on generating mixed-mode flame graphs and I wanted to reproduce those flamegraphs for myself. Setting up the tools is a little bit of work, so I wanted to capture those steps. Check out the Java in Flames post on the Netflix blog for more information.

I've created github repo (github.com/jerometerry/perf)  that contains the scripts used to get this going, including a Vagrantfile, and JMeter Test Plan.

Here's a flame graph I generated while applying load (via JMeter) to the basic arithmetic Tomcat sample application. All the green stacks are Java code, red stacks are kernel code, and yellow stacks are C++ code. The big green pile on the right is all the Tomcat Java code that's being run.


Tools Here's the technologies I used (I'm writing this on a Mac).
VirtualBox 5.1.12Vagrant 1.9.1bento/ubuntu-16.04 (kernel 4.4.0-38)Tomcat 7.0.68JMeter 3.1OpenJDK 8 1.8.111linux-tools-4.4.0-38linux-tools-commonBrendan Gregg's Fla…

Running Sales Demos using Vagrant and VirtualBox

Sales Demo Imagine you have a Java web application that you deploy via a WAR file to a Tomcat server, and you want your sales staff to be able to demo that application from their laptops. The sales folks often times go onsite to customers offices, where they don't have internet access. What options are available for running these kinds of sales demos?

Option #1 - Run Web Server Directly One way to solve this is for the sales staff to install Tomcat and Java on their laptops, and deploy the WAR file by hand. That can work, but if your sales staff aren't that technical, having them configure web servers isn't a good use of their time. Not to mention that if your application requires a database as a dependency, then they must also install a database server such as Postgres, and configure the security settings properly to allow the app to connect.

There's too many moving parts here for sales staff to configure a demo in this way. Clearly, we want to automate this process.