Sizing JVMs and VM Memory

How to estimate the amount of JVM heap and host memory requirement in complex cases where there are more JVM instances per VM and more apps per JVM and have different usage requirement per app ?

let's see some of the metrics to collect to construct an equation on estimating the sizes.

• list the apps per instance. Ideally there will be multiple apps deployed on one instance of JVM and all these apps are not equally used and each app will have its own characteristics thus have uneven memory requirements i.e. based on req. classes, code logic and constructs etc.,

• And identify the 'typical' transactions.. ideally, in any app, 20% of features are executed 80% of times

• Take a first measurement - server start up heap. Once the server is up completely, get the first point after a full GC (hotspot) or first OC (jrockit)

• Now, ideally the first user access is demanding one as it loads lot of stuff. So, do just logins into each of the app and take a note of OCs/FGCs for each user. So, per app, the numbers gives the amount of heap required for first login

• So, XMS - to make it good enough, it can be sum of (server startup heap + sum of all above deltas )

• Now, exercise the typical 20% transaction as single user per app and do not logout - so these are typical active users on the system - call these deltas as h1, h2 etc . This can be on a warmed up server but best is to have no other logged in users in the system. Each delta can be noted or take the final delta after all different app users are in and executed the typical flows but not logged out.

• XMX i.e. the max amount of memory can be XMS + ( total delta * (%app1 conc. users)   +  ... )

for hotspot jvms or other jvms where there are more heap spaces like young/survivor, perm etc., they can be calculated as proportionate ratio of XMX ..

And, typical host memory usage requirement can be calculated as approx 1.8 times of XMX

If someone can simulate load and want to bare the cost, time and complexity of all setup to do that then it is probably best to get the numbers based on test results.., the benefits in doing so can be not just limited to JVM heap but gives the other resource pools across the layers..

Mobiles - The way they transformed made things reachable to some but created complexity to others

I am talking about the quality checks for the applications built for mobile devices. Mobile devices transformation has to be considered as the fastest change in technology so as the complexity in delivering applications on this new platform which is becoming a must !

Besides making sure the features work i.e. testing the functionality, it is the most important task to ensure the app actually perform as expected. someone can say what is there to think about as it is a light weight code .. there is infact a lot that could disrupt the performance - examples - have to support different vendors like android/APPLE, version of operating systems, underlying hardware, resolutions, GPU, CPUs, memory, network carriers and subscribed bandwidths like 2G/3G/4G, inter-apps interruptions, backend thread support for async calls .. what not !

Well there is a lot, I can think of how one can approach to validate performance of the mobile applications and the upstream servers by using some of the great tools in the market.

Below are some of the tasks to evaluate performance and what these tools can do..

• Test and monitor the app's performance on different emulators. If it has to be done independently then each IDE is needed like android studio, IOS IDE and respective skill set as well. Perfecto addresses this with its support to multiple devices 

• Test the app's performance on different real devices- can't do it manually.. can not have a farm of devices.. perfecto can do it using its mobile device cloud.. or its emulator is good enough as well

• Simulate apps performance under different network speeds on devices/emulators - can do by using various softwares like act but again perfecto along with shunra integration can manage devices by emulating various network bandwidths

• Capture the traffic and simulate 1000's of devices load to app servers - this can be done using emulators and tcp capture.. but perfecto-shunra can do and the captured traffic can be fed to Loadrunner. And, Shunra can virtualize networks to emulate load from different locations/carriers/bandwidths during load tests

• At the same time one would have to recheck the app's performance on the device while server under load  - again by perfecto

Its a different world now with mobiles/tablets/wearables that move .. Apps need to support all these !

Gone are those days where apps are accessed from a standalone PC ..

End to end request processing time

In a simplistic view, in a typical online system, this is where one needs to check to know any slowness

Browser rendering time - if the page size too big or too complex with java scripts and style sheets
then the rendering time could take time..
# of static content being used on the web page has an impact on the overall load time. And, if they are not cacheable, the round trips even from a CDN could have an impact on the load time
Total download time for a page depends on how big is the response content and how many susbrequests are triggered part of the page and the network time and entire server side processing time
Page load time is the time by when user sees the page.. so above all have impact on it.

coming to network, it is important to know what is the path the request is traversing from client to the server. Is it taking longest path via CDN or how the addresses being resolved over public internet and any proxy being used etc., the network delay and packet loss are important factors to keep an eye on.

coming to server side, the request processing time depends on many factors.. it depends on underlying infrastructure, resources, architecture, code logic implementations etc.,
but there are few check points to look at to break it down..
checking at http server layer tells the variation between end page load time and total server time. This helps identify any network delays.
checking the difference between http server to app server times tells if there is any delay in the middle layers like authentication routing..
on the app server side, it could spend time in many places.. the routing can happen to many servers or even to external systems. using runtime instrumentation tools, it is possible to break down the time spent in pure code, wait times due to synchronous code blocking, gc pause times, time spent in reading from sockets/wirte to while interacting with DB or in the calls to other servers while making remote calls like rjvm to EJBs or service calls etc., by breaking this way, each underlying activity and the delays can be identified

Each of the above ones are not more than just an index to an ocean of tuneable metrics that each underlying technology modules contains. However, where to look at and what to tune is the key.

Application performance - odd bits

we do normally warm up the systems and concerned about the how it performs under load but will it meet customer expectation or improve customer experience or avoid frustrating scenarios ?
how about the 'first time' and idle case performances ?

did you ever check what is the single user response times and request processing times on server side ? if it is not performing any good for single user then it will not do for multiple. The base performance for single user is what needs to tuned first

what about first time cases ? we can not ask users of the system to hang on until system warms up ! then who will use it first :)  so, analysing cold case system performance is important. it is important to know what happens on first access across the layers in cold cases i.e. after a system restart, app servers or entire VMs or DB etc., and what needs to be tweaked to get better first time performance.

how about sleeping systems ! not all systems work round the clock atleast not all days in a week.. so did you ever check what is the resource usage of idle systems ? there could be some unexpected code path executions might happen even under no load case. So it is important to know the resources usage under no load as they might also run all the time.

how about apps performance under a very slow network access or networks with high packet loss. how reliable are the systems..

what happens when infrastructure fails ? how many users can still continue doing what ever they are doing with out any interruption and feel the same performance of the app while a DR process kicks in..

how about the systems with long running sessions.. users may not logout for long time and it is required to keep the sessions so long and what is the impact !

how the applications handle when majority of end users do not logout but just closes the browser ! how to handle the memory usage in those cases..

Java EE - EJB

Enterprise Java Beans - EJB

EJBs are the Java EE server side components which implement app's business logic. They normally be deployed in EJB containers provided by the app servers. Implementing EJBs can provide scalability to the application and better handling of security and transactions. EJBs can also implement webservices.

Types of EJB:
Session - To implement user actions
- stateful: maintains a state for the client and this bean can not be shared.
- stateless: does not maintain state so can be allocated to any clients. They are reusable. So this
        pool will be less compared to stateful.
- singletone session beans: have only one instance for the whole time.  They can get initiated when the app starts. Although they act as stateless but there is no pool because of single existence

  Message Driven Beans (MDB): for listening to the messages either from queues or JMS
(if someone read about entity beans, they are now part of persistence API..(referring to Java EE7)

Implementation is simple.. infact, the annotations made it so..lets says to write a simple stateless EJB, just annotate the class with @Stateless and implement the business logic.. and to call this EJB, a servlet can do ..ofcourse, you need to annotate with @WebServlet(urlPatterns="/") the url pattern says the context root. And, as usual extend the class with HttpServlet. And, to access the EJB, just annotate with @EJB and then the declare the EJB instance.

However, in a typical EJB implementation, there could be all type of session beans - stateless, stateless bean implementing a service, stateful bean accessed remotely etc.,

A remote interface is required for the beans which allow remote access. This remote business interface defines all the business methods of a bean and annotated with Remote from javax.ejb pacakge and it gets implemented by the session bean. A session bean can be an end point for a web service.

A stateful session bean can have methods annotated with Remove which can be invoked by client to remove the instance.

In the case of singleton bean, the concurrent access from the clients can be controlled in two ways - container managed or bean managed by annotating accordingly. And, the methods must be annotated with the locktype i.e. read or write so that concurrent access can allowed or provided with a synchronous mechanism respectively..

For stateless session beans to implement service end points, they must be annotated with @Webservice and the business methods that are exposed must be annotated with @Webmethods. They can also implement async methods so that clients no need to wait for response from long running methods.

Coming to EJB pools - in weblogic, there is an element called max-beans-in-free-pool in weblogic-ejb-jar.xml. This determines how many EJBs must be made available in free pool. max-beans-in-pool will put a cap on the pool limit. For MDBs, the container will create as many instances required based on the size limited to max-beans-in-free-pool. Default MDB threads are 16 but this can be changed by having custom queue or workmanagers

Oracle VM

Virtualization is a technique to share the hardware resources among multiple systems or users to achieve optimal usage of resources and reducing costs.

Although virtualization is a generic one conceptually, lets talk on server virtualization. This means a bunch of HW resources like CPUs, Memory, Disks, ports etc., are shared among multiple OSs either of same type or multiple type. So, to achieve this we need someone or something to manage the underlying HW and above running guests (OSs). This 'manager' is what is called a 'Hypervisor'.

There are couple of types of Hypervisors
Native or bare metal hypervisor - this is the software which directly runs on host's hardware to control the hardware and monitor the guests OS. so imagine this something that mediates between guest OS and underlying hardware. Example of
such implementation are Oracle VM, VMware EXXi Xen, Microsoft Hyper-V

The other hypervisor is made to run within a traditional operating system and then guest OSs can run on top of it. Example Oracle VirtualBox (which can be installed on an PC where windows is the base OS but virtual box can then host another guest OS like linux..


Oracle VM Server:

This can be installed on X86 instruction set based platforms with Xen hypervisor (GPU licensed) or on SPARC platforms (which will have its own hypervisor).
In general, the above implementation has their own firmware/hardware, a hypervisor and then a super domain/vm which controls the resource allocation to other guest VMs (also called domains or simply guests)

So, simply Oracle VM server is a collection of hardware (CPU, Mem, Network, IO etc.,), hypervisor (for managing underlying baremetal i.e. the hardware), domains (the VMs with thier own set of OS except Dom0 which a complete linux kernel and manages all the other Domains).

Lets explore some interesting things related to Oracle VM

CPU capacity:
how to determine the cpu capacity on a vm server
xm info is the command to use. for example, as shown below, the number of cpus are 72 which are ideally the threads. There are 2 nodes, 18 cores per socket and 2 threads per core
i.e. 2 * 2 * 18 = 72 threads (0 to 71 total, 0-35 on sock1, 36-71 on sock2)

nr_cpus                : 72
nr_nodes               : 2
cores_per_socket       : 18
threads_per_core       : 2

The cpu topology can be viewed by using the commnad xenpm get-cpu-topology
CPU     core    socket  node
CPU0     0       0       0
CPU1     0       0       0
CPU2     1       0       0
CPU3     1       0       0
..
xm info also gives the high level vm details like what bit it supports, what instruction set (like intel x86), number of real cpus, number of nodes, number of sockets, number of threads per core, cpu frequency,
memory, pagesize etc.,
In an hyperthreaded model, each core will run 2 threads instead of one. and this would have counter effects but could improve efficiency..


vCPUs
virtual cpus are the cpus that are assigned to a guest/domu i.e. a virtual machine which runs on a domu can be assigned 10 CPUs which are considered as virtual cpus and the actual bindings to real cpu depends on how they are configured. For example below, vm1 is a virtual machine with the id=1, has 3 vCPUs which are in bind state and mapped to CPUs 3,6 7. This vm1 is configured to have the cpu affinity as 2-35 which is first socket on a 2 socket 72 core machine. so, since there is no absolute binding, the mapping can change in runtime and depends on the availability,
the vcpus can be mapped to any of the real cpus in the range 2-35.

xm vcpu-list
Name  ID  VCPU   CPU State   Time(s) CPU Affinity
vm1   1     0     3   -b-   5354.1 2-35
vm1   1     1     6   -b-   2312.4 2-35
vm1   1     2     7   -b-   2337.8 2-35

you can pin the CPUs for guest vms runtime but to change any affinity to dom0 requires a reboot.. and dom0 always takes the top priority.
And, it is always good to monitor the real cpu usage from the vm server to check how in an  oversubscribed case, the busy vms on the same socket could impact each other..

JNDI - Java Naming and Directory Interface

JNDI - Java Naming and Directory Interface

what is JNDI - its a naming service
why is it needed - in a distributed enterprise application, there are multiple resources like DB pools or business components like EJBs deployed on the Java EE containers and they need a way to locate. JNDI serves that purpose.

Applications can use annotations to locate the resource. Like datasources which are nothing but database resources, provides connection to database. when the application code refers to a
 datasource and invokes JDBC API to getconnection, it gets a physical connection. In case connection pooling is implemented then it gets a handle to pooled connection object instead of direct physical connection.
 These connections need to be closed and when closed will go back to the pool. The pool of database connections will give better performance and better connection handling mechanism.

Similarly JNDI mapping can be done to other services like JMS, LDAP etc.,

Below shown are some of the resources on Glassfish server and the JNDI mapping.