Thursday, 25 February 2021

Configuring Spring Boot Test

In construction.

Spring test annotations: 

@SpringBooTest

@DataJpaTest

@TestPropertySource

@ActiveProfiles

@Sql


@SpringBootTest

It is used for an integration test and loading the entire application context. Through the attribute classes, you may specify the classes and narrow down the scope of the application context. There is another important attribute, webEnviornment, which may specify the type of web application context and also we may turn it off for some tests which do need a web context, so as to lighten the test. 

@TestPropertySource

In some cases, we may need specific properties for a test, by overriding the properties defined in the src resources or by adding new properties. You have two ways to specify test-specific properties.  @TestPropertySource is applied at the class level, and loading a property file for the current test from the classpath by default, or from a specific location. Rather than loading a test property file, using @ActiveProfiles("test")  to activate a specific profile, it may bring in a profile-specific property. meanwhile, It offers an inline way to set properties, to set property-value pairs.  

@SpringBootTest
@TestPropertySource("classpath:mytest-application.properties")
@Slf4j
class LinearRegressionApplicationTests {
@Value("${my.math}")
private boolean myMath;

@DataJpaTest 

It is an annotation that tailors the application context enough to test a repository. All beans above the repository will not be created, but only relevant ones for the current test.  

A feature of @DataJpaTest, it activates an embedded database and disabling other data sources. In this case, if your project by default is using a real database, then you need to disable auto-configuring embedded database

@DataJpaTest
@AutoConfigureTestDatabase(replace = AutoConfigureTestDatabase.Replace.NONE)

@ActiveProfiles("dev") only trigger the bean data-source active. this is why application-dev.properties are not activated by @ActiveProfiles("dev").  

Configuring Spring boot test with different Data Source

When a Spring boot project has multiple data-source, @DataJpaTest may fall in an error, loading application-Context failure. 


@Sql

It is used to annotate the test class or test method, to run a SQL script against a given database. The scripts can be configured to run before the method or after the method. 

@DataJpaTest
@Sql("classpath:insert-data.sql")
@Sql(scripts = {"classpath:delete-test-data.sql"}, executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
class PersonRepoTest {

@Autowired
private PersonRepo personRepo;

@DataJpaTest
@Sql("classpath:insert-data.sql")
@Sql(scripts = {"classpath:delete-test-data.sql"}, executionPhase = Sql.ExecutionPhase.AFTER_TEST_METHOD)
class PersonRepoTest {

    @Autowired
    private PersonRepo personRepo;


@SqlMergeMode

@SqlMergeMode is used to annotate a test class or test method to configure whether method-level @Sql declarations are merged with class-level @Sql declarations. If @SqlMergeMode is not declared on a test class or test method, the OVERRIDE merge mode will be used by default. With the OVERRIDE mode, method-level @Sql declarations will effectively override class-level @Sql declarations.

@SqlConfig

@SqlConfig defines metadata that is used to determine how to parse and run SQL scripts configured with the @Sql annotation. The following example shows how to use it:

configure different data-source for test


@ContextConfiguration

@ContextConfiguration loads an ApplicationContext for Spring integration test. @ContextConfiguration can load ApplicationContext using XML resource or the JavaConfig annotated with @Configuration. The @ContextConfiguration annotation can also load a component annotated with @Component, @Service, @Repository etc.



Monday, 22 February 2021

Java File IO

Java Old IO

File IO before JDK 7.0 is defined as old File IO, in contrast with the NIO.   

In the old IO, the major concept is the File, and stream, reader/writer, buffer these facilities are used to wrap the File and achieve the extended functionalities. 

IO File

In the old IO, both the path and file are represented by the File class; and service functionalities like delete(), createNewFile(), mkdir(), exist() are included in the File class. File stands for abstract File, it is not associated with a physical file or path.  After the creation method is invoked, it is pointed to this physical file and path. When a File created corresponding to an existing file, then it is automatically associated with it, otherwise, the newly created File instance is just an abstraction.  

File(String fileName)
File(String pathName) 
File(String parentPath, String childPath)
File(File parent, String child) 
File(File path, File fileName) 

Types of  File Data

Java IO handles two types of data, i.e. byte and character.
Byte: read or write 8-bit to or from the data source, mainly zip, image and PDF file.
Character: read or write 16-bit to or from the data source, mainly uni-code text file.

Types of File IO Handlers

Java IO handles input and out respectively:
For byte stream via FileInputStream and FileOutputStream.
For character stream via FileWriter and FileReader.

Buffered Streams

File IO uses the decorator pattern to manage how to extend File IO functionalities.  

Java IO uses external Buffered Input/Output streams(byte) or Buffered Reader/Writer(char) to add the buffer functionalities on the existing byte IO or char IO. 
Buffered streams are commonly used for achieving a better performance. Invoking system IO operation to read or write a single byte or character is not efficient. So, buffered IO was introduced. It stores byte or character to be read or written into a piece of memory space(buffer). When a buffer is full or empty,  IO API is invoked, so that the buffered data can be handled like a batch job. 

The following is a simple example that explains how to implement buffered stream operations.

public class BufferedStream {

    public static void main(String[] args) throws IOException {
        String content = "Java is verbose!";
        try (BufferedOutputStream bos = new BufferedOutputStream(new FileOutputStream("my.txt"))) {
            bos.write(content.getBytes());
            bos.flush();
        }
        
        try(BufferedWriter bw = new BufferedWriter(new FileWriter("my2.txt"))){
            bw.write("Java has some improvements");
            bw.flush();
        }

    }
}

Read and Write Byte to File
Abstract class InputStream and OutputStream its sub-implementations
FileInputStream, BufferedInputStream, DataInputStream, and ObjectInputStream FileOutputStream, BufferedOutputStream, DataOutputStream, and ObjectOutputStream

public class CopyPdfUsingStream {

    public static void main(String[] args) {

        String name = "billet.pdf";
        String copy = "copyBillet.pdf";

        try (
                FileInputStream fis = new FileInputStream(name);
                FileOutputStream fos = new FileOutputStream(copy);) {

            int data;
            while ((data = fis.read()) != -1) {
                fos.write(data);
            }

        } catch (IOException ex) {
            Logger.getLogger(CopyPdfUsingStream.class.getName()).log(Level.SEVERE, null, ex);
        }

    }

}

Java NIO

Path, Paths, Files are classes of Java NIO since Java 7. The Path is a replacement of File in the NIO. Paths is a utility providing static factory method to create Path objects. Files is another utility class,  manipulating a File object.  

Path Object

The Path is an Interface representing a directory or a file, but independent of a physical file system. Please note that Path is an abstraction of a file or directory. 

Why we need an Abstraction of Path? It because the underlying OS has different directory delimiters or different directory roots like in Win OS and Unix OS.  

Paths Class

Paths class creates a Path object. 

Paths. get(String first, String... more);  
FileSystem. getDeafault(). get(String first, String ... more)

The above is the most simple and straightforward way to create a Path object, for you don't need to worry about the separator format, forwarding slash or backwards slash.

Once Path object is created, it is system-dependent; and two Paths having the same attributes but 
for the different OS, they are different Path object.  

Path and File can be converted to each other. 

Resolving a Path
path1 resolve path2: resolve path2 within path1's directory, meaning that joining path2 on the top of path1. joining a relative path on the top of an absolute path; or joining a relative path on the top of another relative path; joining an absolute path on the top of a relative path, doesn't make sense, because an absolute path doesn't depend on any other path, so java will return the absolute path again.

Relativizing a Path
path1 revitalize path2: give me a path that shows how to get from path1 to path2, meaning that finding out the difference between the two paths. relativizing an absolute path to a relative path, or relativizing two relative paths. 

Files Class

Files offer static methods to operate a Path instance, like determine existence, copy, delete and moving around. 
 
Files.exists(Path) and Files.notExists(Path)
Files.copy(sourcePath, targetPath): if source is not existed, it throws FileNotExistedException; if the targetPath is already existed, it throws FileExistedException. 

File Attributes

You use the file to store data and OS store additional data about your data about creation, modification and accessing so on.  You can get individual attribute, or access a group of attributes so as to improve the performance. 

Accessing individual attributes

Files class defines static methods to access individual attributes. 
Files.size(path)
Files.isDirectory(path)
Files.isHidden(path)
Files.getLastModifiedTime(path)
Files.isReadable(path)
Files.isWritable(path)
Files.isExecutable(path)
Files.isSameFile(path)

or 
Using Files class getAttributes(String nameOfAttribute)

Modifying file/directory attributes

using Files class static methods

Set the value of a file attribute
Path setAttribute(Path path, String attribute, Object value, LinOption ...options) 

Updates a file's last modification time
Path setLastModificationTime(Path path, FileTime time) 

Updates the file owner
Path setOwner(Path path, UserPrincipal owner) 

Accessing group attributes

Instead of accessing individual attributes, you may access a group of attributes. In this way, it may improve accessing performance.

Attributes are classified as groups, for different file systems may offer different attributes. An interface represents a group of attributes.  

There are two types of attribute groups, one is for reading attributes, another is for modifying attributes.

The term “View” are collectively referred to as the view interfaces, and they’re used to update file and directory attributes. The interface without “View” is used to read out the attributes.
  • BasicFileAttributes and BasicFileAttributeView: all file attributes supported by all OS.
  • DosFileAttributes and DosFileAttributeViewDosFileAttributes extends BasicFileAttributes, it defines file attributes supported by DOS.
  • PosixFileAttributes and PosixFileAttributeView: PosixFileAttributes extends BasicFileAttributes, it defines file attributes supported by UNIX. 
  • AclFileAttributeView: exitends form FileOwnerAttributeView; only for the Windows OS; handling file permissions
  • FileOwnerAttributeView: get owner and set Owner. 
  • UserDefinedFileAttributeView: access or modify user-defined attributes to a file or directory; for instance, setting a File/directory a delete attribute, later on removing them collectively. 
Reading attributes:

As reading file/directory attributes, providing the class type of target interface.

DosFileAttributes dosFileAttributes = Files.readAttributes(path, DosFileAttributes.class);
System.out.printf("DosFileAttributes: isArchive? %b \n ", dosFileAttributes.isArchive());
System.out.printf("DosFileAttributes: isSystem? %b \n ", dosFileAttributes.isSystem());

static Map<String,Object> readAttributes(Path path, String attributes, LinkOption... options)
throws IOException

attributes: it can take values basic, dos, and posix. If this value is absent, it defaults to basic. You can specify * to read all the attributes for a group.


Updating attributes

if you want to modify file attributes, then achieving view instance first. Afterwards, via the view, you may reset the attributes.  

//BasicFileAttributeView interface defines API to modify attributes
BasicFileAttributeView view = Files.getFileAttributeView(path, BasicFileAttributeView.class);

//basicFileAttributeView modify all timeStamps
long now = System.currentTimeMillis();
FileTime creationTime = FileTime.fromMillis(now);
FileTime accessTime = FileTime.fromMillis(now - 1000);
FileTime modifyTime = FileTime.fromMillis(now - 2000);
view.setTimes(creationTime, accessTime, modifyTime);











Friday, 12 February 2021

Generics and Collections

Java Generics and Collections

Collections are about how to store and manage elements in a data structure. It includes List, Set, Queue, and Map. Generics is about how to maintain element type in collections to be type-safe.  That is the reason why these two topics are always staying together. 

Generics

Java generics are introduced from Java 5. The purpose of using generics is to enable you to mark your intent of using a class, method, or interface with a particular data type. Generics add compile-time safety to collections. By this way, a type inconsistency may be found in an early time, rather than in the run time. Generics can be applied to a class, an interface, a method, and even a constructor. They are called generic classes, generic interface, generic methods, or generic constructors.

The key for Generics is to declare type parameters, it is a placeholder that is replaced by an actual type in the compiling time. For an instance, List<String> is a parameterised type, which stands for a list that contains String alone, then the instance must host String alone, i.e new ArrayList<String>(). 

Generic type parameter declaration 

Type parameter follows a naming convention, i.e. <T>, a single capital character.  Type parameters can be declared with a class, a constructor, an interface, or a method.   

For a generic class/interface, it declares its generic types parameter just after its class/interface name;
For a generic method,  it declares its own type parameter just after the access modifier before its return type.  This is important, for otherwise, the compiler may don't know what return type it is.

Type argument → Type parameter; when compiling using actual type to replace the type parameter.
The compiler may replace type parameters using actual types presented in generic type argument, or infer it from the real data types.

For class(interface) generics: a type parameter is inferred from a parameterised type, like new List<Integer> ints = ArrayList<>(); 

For method generics: type parameter is inferred from the real data type, from the input arguments or returned value type; 

Type erasure in the runtime

After compilation, the generic type will be replaced by an Object type or its generic boundary class type. The compiler generates a single version of a compiled class file for the generic class/interface, but not one for a generic class or one for a raw class. So during the runtime, there are no generic types. Java Generics is a compiling time check, to verify the object types are consistent in the collection as we intend to. 

Bounded generic types

The purpose is to restrict types that can be used as a type-argument by putting a boundary.  
A type parameter can have multiple boundary types, i.e. one class type and many interface types. It means that a type-argument must satisfy all types of constraints.

Upper-bounded generic type: 
<T extends Number>: meaning that type argument is a subtype of Number type.

Lower-bounded generic type: 
<T super Number>: meaning that type argument is a supertype of Number type. 


A generic class can extend another generic class or interface. 

A generic class may extend from another generic class, but it must pass a type argument to the type parameter of its base class. Otherwise, it won't be compiled.

Class ArrayList<E> implements List<E>
The type parameter E is passed to its base class as a type argument.

The inheritance doesn't apply to generic arguments. 

A rule: Reference generic type must be identical to the instance generic type. 

List<Object> objects = new ArrayList<Integer>(); 

Although ArrayList is-a List, and therefore an ArrayList instance can be assigned to a List reference variable.  However, this concept doesn't work with generic arguments.  The above won't compile. 

I think the reason because this violates the purpose of the generics, checking the consistency of the element types.

List<Object> objects = new ArrayList<Object>(); 
//the above compile correclty.

Non-generic class or interface extends from a generic class

A non-generic class can extend another generic class. However, it needs to pass type argument(a type of instance)  to its base class.
class Parcel<T> {}
class NonGenericPhoneParcel extends Parcel<Phone>{}

A non-generic class can implement a generic interface by replacing its type parameters with actual types. The same as the above

Using wildcard

A wildcard(?) type parameter stands for any object types, or say a unknow type. A wildcard type parameter can be constrained by an upper boundary or a lower boundary.

List<?> list = new ArrayList<String>(); //this is valid
List<Object> list = new ArrayList<String>()//this is not valid. must be identical. 
So the wildcard stands for any type of objects alone.

Adding objects to a wildcard defined collection is not allowed.
List<?> list = new ArrayList<String>(); //this is valid
list.add(new Book()); //this won't complile

The above List<?> accepts any type of objects; Compiler, therefore, cannot make sure of type consistency. So it simply prevents it from compiling.  However, if we use it as a method argument, then it gives a sort of flexibility to accept a group of object types, meanwhile, you get a data consistency protection because in the method the wild card collection cannot be modified. 

Using a wildcard as a method argument

void method(List<?> list){
//you can traverse all elements of input list.
list.forEach(System.out::println);
//but you cannot add element to it.
list.add(new Object())//compiling error here
}

However, you lose the possibility to add a new element within the method; it will cause a compilation error. 

Upper-bounded wildcard

For upper-bounded wildcard, you can read over the collection, but you cannot add an element. The problem is because an upper-bounded wildcard is open for all sub-types of the upper bound, so compiler prevents adding a new element. 

List<? extends Gift> list = new List<Gift>(); //valid; 
List<? extends Gift> list = new List<Book>();//valid
List<? extends Gift> list = new List<Follower>();//valid

However, you cannot add a new element in the lists mentioned above; it causes compilation errors. 
It is good for the method input argument, which makes the method accepts a group of object types. 

void method(List<? extends Number> list){
list.forEach(System.out::println);
}

Lower-bounded wildcard

For lower-bounded wildcard, you can add a new element, i.e. boundary type and its subtypes; you can loop over each element, but via Object type.  Actually, it sounds weird. 

<? super Type> a supertype of a class type, including the boundary.
List<? super Gift> list = new ArrayList<Gift>();
List<? super Gift> list = new ArrayList<Object>()

You can add new element Gift type and its subtypes. 

for(Object gift:list){}

List<? super Gift> list; read as Object, add Gift and its subtypes. 


Collection

The collection is a data structure, to hold a group of objects(elements); and data structure functionalities, like add, insert, remove, sorting, etc.

Set, List and Queue Interface extends from Collection Interface extends Iterable Interface. Therefore, Set, List and Queue are both collections and iterable. The map doesn't implement Collection interface, but it is a part of java Collection and considered as a kind of Collection.

Set maintains a collection of unique elements.
List maintains a collection of elements, may be duplicated.

Concrete implementations:
Set: HashSet(not ordered, sorted by hashcode), LinkedHashSet(ordered), and TreeSet(sorted by comparable)
List: ArrayList(ordered) and LinkedList(ordered)
Queue: LinkedList(ordered) (LinkedList implements Queue interface)

Ordered: retrieving elements in order as they are added. All lists are ordered, but not HashSet and TreeSet. When needing an ordered set, then implementing LinkedHashSet. It is internally implemented as a linked list.

Sorted:  elements are sorted, and they may need to implement Comparable interface, like TreeSet. Otherwise, the client has to provide a comparator as instantiating it. Please note: HashSet is implemented as HashMap, it was sorted by the hash code of an element, which is translated into an index number used as a key value of the map. That is why HashSet has no guarantee of order but sorted by the hashcode.

ArrayList supports random access, for internally it is implemented by an Array so that it supports accessing element directly by array indexing.  LinkedList supports sequence access, for underlying, it implemented a double-linked list, each node(holding the elements)  contains pointers pointing to the previous element and next element. When accessing one of the elements of LinkedList, it has to traverse from the head to the tail, until finding a matched element.

HashSet retrieving uses hashcode, in some sense it is a kind of indexing, but invoking hashcode-index calculation, and when more than one elements sharing the same index, it may invoke equals method to locate the element. 

TreeSet →NavigableSet→SortedSet→Set
TreeSet is sorted in natural order, or by an external custom comparator. TreeSet offers navigable and sorted APIs. 

ArrayList vs LinkedList:
ArrayList is internally implemented as an array, so it supports random accessing. However, its array size is not mutable, it needs frequently re-instantiate a new array according to current array size and capacity factor, and also resulting in extra unused memory space. In a linked list, this can be easily by reset the node previous and next pointer.

LinkedList →Dequeue→Queue 
LinkedList implements Deque that implements Queue. LinkedList is, therefore, be used to implement Queue and Stack. 

PriorityQueue  → Queue
The purpose of PriorityQueue is First priority first out. PriorityQueue is sorted according to element natural order or an external comparator. 

From the view of insertion and removal, the linked list gives much better performance for implementing a Queue. It doesn't need to swap the buckets but simply cutting off a pointer and re-pointing to a new bucket.   




Map

The map is packaged within the collections.  The Map is an interface. It is a type of collection, although it doesn't implement the Collection interface. 

HashMap: 

The Map is ordered by the key hashcode, which is translated into an Index pointing to one bucket of the map. So the Map will rearrange the order of the elements.

boolean containsKey(key)
boolean containsValue(value)
V get(Object key): get value from its key
Set<K> keySet();
Collection<V> values(); return a collection of values
remove(key)
clear()
putAll(Map<? extends k, ? extends v> map)

LinkedHashMap: (ordered)

keeping insertion order. 

TreeMap 

TreeMap implements the NavigableMap interface, which extends the SortedMap interface. TreeMap is sorted in its natural order(implemented Comparable interface), or sorted according to any external custom comparator in many ways.

SortedMap 

SortedMap is an interface, offering APIs taking use of sorted elements; for instance, you may easily Finding the min. and max. from a sorted map. 
K firstKey();
K lastKey(); 

Copying a portion of Map from fromKey(included) and toKey(excluded).
SortedMap<K, V> subMap(K fromKey, K toKey):  >=fromKey AND <toKey

Copying a portion of Map,  whose keys are strictly less toKey
SortedMap<K,V> headMap(K to): <

Copying a portion of Map from the tail, whose keys are strictly greater than fromKey
SortedMap<K,V> tailMap(K from): >


NavigableMap offers APIs to search key-value, like  lower(), floor(), higher(), and ceiling() 
lowerKey(Key): < key
floorKey(Key): <=key
higherLey(Key): > key
ceilingKey(Key): >=key

descendingMap(): returns a NavigatbleMap in a reverse order

pollFirstEntry(): return and remove the first entry.
pollLastEntry(): return and remove the last entry.




Can Jackson Deserialize Java Time ZonedDateTime

Yes, but must include JSR310. Thus ZonedDateTime can be deserialized directly from JSON response to POJO field. <dependency> <g...