Sunday, 22 December 2019

JVM Memory Management

Escaping Reference

Java passing variable to a method by a copy of variable or a copy of the reference that points to the object located in the heap. If exposing a reference, meaning that the instance may be musted unexpectedly. thus we need to escape reference.

Garbage
Stack: reference; (stack overflow exception)
Heap: instance   (running out memory exception)

when an instance in heap has no reference pointing to it, then this instance is eligible garbage.

Analysing Tool
Goto JDK bin folder, find the application 'jvisualvm' and run it.
It is a visualizing tool to show heap memory (used and unused).

The curve used heap memory may indicate the trend of memory leak.

Garbage Collection Process

java may automatically free memory as objects are not used any more.
it is called 'mark and sweep', consisting of two processes; 1) marking instances that need to be retained and 2) then sweeping those that no needed any more.

Before implementing this process, all threads need to be stopped. (this is the downside of GC)
After that, Checking the references on the stack; marking the instance it is pointing to, and the instances it depends on.

Heap Generations

For solving GC freezing(all threads need to be stopped as carrying out GC.)
Heap memory is classified into the young generation and the old generation.
If an object survives from a GC, then its life cycle may last until the end of the application.

the new object is created in the young generation space. (minor collection)
the surviving object is copied into the olde generation space. (major collection)

PermGen and MetaData

Java 6
Heap memory:
the young generation, old generation, and permanent generation(string and class metadata)

Java 7
the string is moved to the old generation.

From Java 8
class metadata is moved to a separate space, called metaspace, which is not part of heap memory any more. meaning that it has no size limitation any more.

Tunning JVM
setting up the max heap size: -Xmx512m;  start heap size: -Xms125m
java doc shows default heap size = 256mb

Before java 8, PermGen has been removed from java 8
Max PermGen Size: -XX=256M

-verbose:gc   print out to console when a GC takes place
-Xmn256m  set young generation size
-XX:HeapDumpOnOutOfMemory creating a heap dump file.

different type of garbage collector
-XX:+UseSerialGC  serial: using a single thread :
-XX:+UseParrelGC  parallel:  using multiple threads
-XX:+UseConcMarkSweepGC
-XX:+UseG1GC
                             


Sunday, 13 October 2019

Could not extract response: no suitable HttpMessageConverter found for response type [class dk.enettet.evu.core.model.Address] and content type [application/octet-stream]


Could not extract response: no suitable HttpMessageConverter found for response type [class dk.enettet.evu.core.model.Address] and content type [application/octet-stream]
org.springframework.web.client.RestClientException: Could not extract response: no suitable HttpMessageConverter found for response type [class dk.enettet.evu.core.model.Address] and content type [application/octet-stream]
at org.springframework.web.client.HttpMessageConverterExtractor.extractData(HttpMessageConverterExtractor.java:121)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:994)
at org.springframework.web.client.RestTemplate$ResponseEntityResponseExtractor.extractData(RestTemplate.java:977)
at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:737)
at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:670)
at org.springframework.web.client.RestTemplate.getForEntity(RestTemplate.java:338)
at org.springframework.boot.test.web.client.TestRestTemplate.getForEntity(TestRestTemplate.java:275)
at dk.enettet.evu.security.AuthenticationAndAuthorisationTest.given a wrong username but a right password resulting in an authentication failure(AuthenticationAndAuthorisationTest.kt:26)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:532)
at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$6(TestMethodTestDescriptor.java:171)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:167)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:114)
at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:59)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$4(NodeTestTask.java:108)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:98)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:74)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$4(NodeTestTask.java:112)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:98)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:74)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:38)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$4(NodeTestTask.java:112)
at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:72)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:98)
at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:74)
at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:32)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:51)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:220)
at org.junit.platform.launcher.core.DefaultLauncher.lambda$execute$6(DefaultLauncher.java:188)
at org.junit.platform.launcher.core.DefaultLauncher.withInterceptedStreams(DefaultLauncher.java:202)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:181)
at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:128)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:102)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:82)
at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:78)
at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
at com.sun.proxy.$Proxy2.stop(Unknown Source)
at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:132)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:175)
at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:157)
at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
at java.lang.Thread.run(Thread.java:748)


MockMvc: testing  from the server side, simulated servlet dispatcher, but not a container. 


RestTemplate: testing like from client side;

TestRestTemplate:



























For my case, the error is due to the response entity data type is not correctly defined; so it confused the json converter. It received an object type A, but it is required to be converted to type B. So it may throw this specific exception.

@Testfun `given right username and password, but not allowed to access this resource, resulting in an authorisation failure`() {
    val response: ResponseEntity = testTemplate.withBasicAuth("testName1", "test1")
            .getForEntity(resource, String::class.java)
    assertThat(response.statusCode, `is`(HttpStatus.FORBIDDEN))
}

@Testfun `given right username and password, but allowed to access the resource, resulting in an ok status`() {
    val response: ResponseEntity

Friday, 11 October 2019

Tuesday, 8 October 2019

Handling Security Exceptions in Spring Boot


Actually, handling exceptions thrown in the security is different from handling ones thrown from the rest APIs, for they happens in the filters before reaching the Servlet dispatcher.

Hence, we cannot handle security-relevant exceptions in @ControllerAdvice and @ExceptionHanlder,  it needs a different way


Encoding Issue Running Gradle Test

Gradle setup: if having a problem at the command line

It may due to missing encoding set in the Gradle property.

We may setup the encoding convention to force the JVM using a specific encoding strategy.

org.gradle.jvmargs=-Xmx2g -XX:MaxMetaspaceSize=512m 
-XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8

Setup Gradle Properties

JVM Configuration Attributes

Intellij setup: if having a problem at the IDE

Intellij  GOTO 'Help' Tab, Click 'Edit Custom VM Option'

adding

-Dfile.encoding=UTF8






Can Jackson Deserialize Java Time ZonedDateTime

Yes, but must include JSR310. Thus ZonedDateTime can be deserialized directly from JSON response to POJO field. <dependency> <g...