Maven Review

Maven is a project management tool, which is represented a the Project Model Object(POM).

Project Coordinate

Coordinate of the maven project is used to uniquely identify a maven project, and locate it. compulsorily, it requires the followings: group id: com.xxx; artefact id: project name;
version: version control.

group id: is used to identify the development team.

artefact id: is a short name to identify the project and code.
project name: the name of the project.

Parent POM

Parent POM is independent and doesn't belong to any project. It stands at a high level and may be used for an organisation to provide common versioned dependencies and plugins. It behaviors like an abstract class. It provides control on the versioned artefacts and licensed dependencies in one place.

Parent pom must be packaged as a POM.

Effective POM

Maven project is set up with default behaviors. It because Maven POM automatically extends from a super POM; the following command may show the effective pom file.

mvn help:effective-pom

Maven Project Structure

Maven project has a fixed file structure to follow.
The POM file is located at a project root.

src folder:

The source is located at the src/main
Java code is located at the src/main/java

Resources are located at src/main/resources
The test is located at the src/test.
Java test code is located at the src/test/java
Test resources are located at src/test/resources

target folder:
used to take all compiled results.

Build Lifecycle

The Lifecycle consists of many different phases and defines the processes for building and distributing a project. Maven plugin goals are attached in maven life cycles. Meaning that a big project management process is carried out by plugin goals respectively. For a developer building a project, this means that it is only necessary to learn a small set of commands to build any Maven project and the POM will ensure they get the results they desired. 

• validate: is all the necessary information for this project available? 
• compile: compile the source code of the project, and put them in the target folder.
• test: test the compiled source code using a suitable unit test framework.
• package: take the compiled code and package it in its distribution format, fx: jar. 
• verify: run integration tests to ensure quality criteria are met. 
• install: install the package into the local maven repository for other project's dependency. 
• deploy: copying the final package to the remote repository, for sharing with other developers. 

The default Maven lifecycle is listed as the above. there are other life cycles.

Plugins

The plugin is the reusable common build logic across different projects. All the tasks like compiling code, testing them with JUnit, packaging files, and documentation are carried out.  Maven main functions are carried out by the plugins.

each maven plugin may have several goals

mvn maven-plugin-name:goal

Type of Maven plugins: 

• Build plugins: defined under the <build>
• Report plugins: defined under the <reporting>

Archetype

The archetype is a Maven project template. A developer may create a project base on an archetype.

mvn archetype: generate

maven-plugin: archetype; and goal: generate

Using maven archetype plugin to generate a web app project skeleton from a command line.

mvn archetype:generate -DgroupId=com.ynz -DartifactId=sampleWeb -DarchetypeArtifactId=maven-archetype-webapp -DarchetypeVersion=1.4 -DinteractiveMode=false

Package

package information is stored in a pom file. 

package types: jar, war, and ear. 

default package is a jar if a type is omitted. 

Dependency Scope

• Compile a default dependency scope. The dependent jar is available in all the stages.   
• Test: The dependent jar is available during the test. It is designed to reduce the size of the artifact. 
• Import: indicates this dependency(like BOM) will be replaced by another POM.

Force Repository Update

When Maven executes this pom.xml file, the dependencies will be downloaded from a central Maven repository and put into your local Maven repository. If we already have the dependencies in our local repository, Maven will not download them.

The download happens when we execute the following commands:

• mvn package
• mvn install

Both of the above include executing the following command:

• mvn dependency:resolve 

We can solely resolve the dependencies by just running the command above.

Corrupted Dependencies

A network glitch while downloading dependencies, may cause corrupted dependencies.  The maven will feedback an error message. 

Could not resolve dependencies for project ...

As we know, maven won't download an existing dependency, so we may force download all dependencies, by attaching a command option, -U. 

• mvn package -U
• mvn install -U
• mvn dependency:resolve -U 

In some cases, we need a deeper command, purge a local repository: 

mvn dependency:purge-local-repository

Analyse Maven Dependencies

• mvn dependency:analyze
• mvn dependency:tree

Some of the information provided by the dependency:analyze goal includes:

• Dependency convergence issues: When different versions of the same dependency are used in the project, which can lead to runtime errors.
• Unused dependencies: When a dependency is declared in the project but is not actually used by any code.
• Missing dependencies: When a dependency is required by the project but is not declared in the pom.xml file.
• Dependency conflicts: When multiple dependencies declare the same artifact with different versions.

Mediation Principle

Maven uses the principle of Dependency Mediation to resolve conflicts between different versions of the same dependency. Dependency mediation is the process by which Maven selects a single version of a dependency when multiple versions are available in the dependency tree.

The basic principle of dependency mediation is to select the "nearest" version of a dependency, i.e., the version that is closest to the root of the dependency tree. This means that if two different versions of the same dependency are declared in different parts of the dependency tree, Maven will choose the version that is nearest to the root of the tree and use that version for the entire tree.

For example, suppose your project depends on two libraries, A and B, and both libraries depend on the same library, C, but with different versions. Library A depends on version 1.0 of library C, while library B depends on version 2.0 of library C. In this case, Maven will select the version of C that is closest to the root of the dependency tree and use that version for both libraries A and B. If library A is closer to the root, then version 1.0 of library C will be used for both libraries, and if library B is closer to the root, then version 2.0 of library C will be used for both libraries.

In addition to selecting the nearest version of a dependency, Maven also supports the concept of Dependency Exclusion. This allows you to exclude specific transitive dependencies that are causing conflicts in your project. You can exclude a dependency by adding an exclusion element to the dependency declaration in your pom.xml file.

Overall, Maven's dependency mediation principle helps to ensure that your project has a consistent set of dependencies that are compatible with each other, even when different versions of the same dependency are used in different parts of the project's dependency tree.

Skipping Tests

To skip running the tests for a particular project, set the skipTests property to true.

<project>
  [...]
  <build>
    <plugins>
      <plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-surefire-plugin</artifactId>
        <version>3.0.0</version>
        <configuration>
          <skipTests>true</skipTests>
        </configuration>
      </plugin>
    </plugins>
  </build>
  [...]
</project>

You can also skip the tests via the command line by executing the following command:

mvn install -DskipTests

If you absolutely must, you can also use the maven.test.skip property to skip compiling the tests. maven.test.skip is honored by Surefire, Failsafe and the Compiler Plugin.

mvn install -Dmaven.test.skip=true

 

Reference: 

Super Pom

Linux

Linux principles:

• everything in Linux is a file
• small a single purpose programs
• chain programs together to perform a complex purpose
• avoid user interface
• configuration in a text file

Terms:

Shell =>Kernel=> Hardware

Bash is a sort of Shell, other shells, fx: Fish, Zsh, Ksh

A Terminal, also called a shell, is a GUI to interact with the kernel.

Linux folder

/(forward slash): Root 

/home/username: each user has a personal folder

/bin: command binaries. 

/etc: local system configuration files

/lib: shared libs shared for system boot

/opt: the third-party tools

/var: variable files, fx: log files

Linux commands

ls -a: list all files including hidden ones.

ls -l: list files in a long format.

ls -l *.txt

 

cd: change the current directory

.. indicates the parent folder

mkdir: make a new folder

mkdir folder1 folder2: creating several folders

mkdir -p folder/nested-folder: creating m-nested folders

rmdir: remove a folder

rm: remove one file or multiple files

rm blue red yellow

rm -v blue red yellow 

-v: verbose.  -r: recursive -i: interactive -f: force

rm -rf folder-name

touch: if the file is not existed, create one, else update to a new timestamp. 

touch new-file-name

open: open a file or directory just like double-clicked file icon. 

open. open the current folder in a graphic presentation

open file-name: open the file in OS default text editor

> : redirect output console

date > newFile.text: output current date to designated file, and overwrite the file entirely.

>>: redirect out the console and appending

date>> newFile.text: out current date to designated file, and appending. 

cp: copy file or folder

less: a UI showing the entire file, scrolling down and up.

less file-name

WC: word count

wc -l file-name: shows line number in a file

wc -w file-name: shows how many words are in a file

piping |: taking one command output as input of another command. 

Env var

expending dollar sign or tilde

dollar sign path

echo * all path names

echo *.txt

? matching each single character

cry0l1t3@htb[/htb]$ ls -l /etc/passwd

- rwx rw- r--   1 root root 1641 May  4 23:42 /etc/passwd
- --- --- ---   |  |    |    |   |__________|
|  |   |   |    |  |    |    |        |_ Date
|  |   |   |    |  |    |    |__________ File Size
|  |   |   |    |  |    |_______________ Group
|  |   |   |    |  |____________________ User
|  |   |   |    |_______________________ Number of hard links
|  |   |   |_ Permission of others (read)
|  |   |_____ Permissions of the group (read, write)
|  |_________ Permissions of the owner (read, write, execute)
|____________ File type (- = File, d = Directory, l = Link, ... )

chmod 777 file-name : change file read-write-executable

find . -name fileName: find the current folder by name 

which: shows the path to a file

which file-name: locate a file

ps: shows all processes

ps -ef: extended format 

ps -ef | grep bash

kill PID: kill by its process ID

top: show statistics of system usage

wget URI: download resources

su: switch user to log in as someone else

exit: log out a user account

sudo: super user do, run a command as a root user

sudo apt update:  update the APT package index in line with the repository.

sudo apt upgrade: update the installed packages to the latest version.

sudo apt upgrade specific-package: update an installed package to the newest version.

sudo apt install package-name: install package 

source file-name [argument]: shell built-in command that can execute a file containing a list of shell commands. 

export: built-in shell command to mark env variable to export the child processes(change visibilities). 

sh xxx.sh: execute a script file

Vi/Vim Editor

vi target_file_name

entry mode| edit mode | command mode

press esc, and then press key i, then seeing bottom showing 'Insert'; now in an edit mode

press esc, then quit from insert model, then in an entry mode

press esc, then press :, then enter command mode.

press esc, bottom right input commands

:w save current file

:q quit from file without saving

:wq  write and quit from current file

:q! force to quit without saving

:set nu showing line number

:set nonu turning off line number

Network

ping domain_name or ip_address

Check Remote Ports are Reachable Using ‘nc’ Command

Using netcat, you can check if a single or multiple or a range of open ports as follows. The command below will help us see if the port 22 is open on the host 192.168.56.10:

nc -zv 192.168.1.15 22

-z – sets nc to simply scan for listening daemons, without actually sending any data to them.
-v – enables verbose mode.

The next command will check if ports 80, 22 and 21 are open on the remote host 192.168.5.10 (we can use the hostname as well):
nc -zv 192.168.56.10 80 22 21

and it is also possible to check a range of ports.

nc -zv 192.168.56.10 20-80

how to install JDK from Linux 

Concurrent Programming Review

Process and Thread

The process is self-contained. It doesn't share the resource with others. A thread is a minimal execution unit, living inside a process. One process may consist of many threads. Threads may share a common resource with each other.

Volatile: Solving the visibility issue between threads.

Each Thread has its own local cache, so the variable defined in a thread won't be visible by another thread. When using Volatile keyword to modify a variable, the OS will push the variable into the shared cache(among the threads), thus it becomes visible to all other threads.

Volatile provides visibilities between isolated threads, however it doesn't provide an atomic protection from multiple step operation(read-modification-write).

Concurrent and Parallel computing 

Concurrent computing task can be decomposed into sub-tasks, which are order-independent.  These sub-tasks are carried out at the same core but different threads. Form an external view, it seems that these tasks are carried out simultaneously, but actually not.

Parallel computing needs parallel computing hardware first, i.e. having more than one core. The computing task is carried out in independent cores simultaneously.  

Mutex

Mutex means mutual exclusion is the way to solve the data race. At any time, only one thread is allowed to access the shared data. 

Data race: 

Simultaneously multiple threads access a common memory location.  The data stored there may be not consistent and being polluted. Ideally,  all steps should be carried out in a single uninterruptible operation, so as to prevent data from pollution. 

Lock: 

Two types of locks, i.e. extrinsic or intrinsic lock.

An extrinsic lock is explicitly declared from Lock interface and its implementations. A critical section is protected by an explicit lock and afterwards, the lock is unlocked.

Synchronization: 

Two types of synchronization, i.e. synchronization method or synchronization statements.

Every Java class or object has an intrinsic lock associated with it. The intrinsic lock must be achieved first, so as to access the synchronized method or statements.

Note: ➀watching out Synchronizing on the class or instance; ➁ don't synchronize on an immutable object, for it generates a new instance after its value being modified.

Atomic variables:

The most commonly used atomic variable classes in Java are AtomicInteger, AtomicLong, AtomicBoolean, and AtomicReference. These classes represent an int, long, boolean and object reference respectively which can be atomically updated.

For instance, AtomicInteger provides a method incrementAndGet(), it is not synchronized, but it offers
atomicity guarantees by using Low-level compare-and-set(CAS) instruction. 

Try Lock:

The tryLock method offers a non-blocking way to acquire the lock(explicit lock), so as to improve performance.

When a thread wants to acquire the lock, but not available, it may go away to carry out some other tasks, rather than being blocked and waiting there. This is the purpose to have 'try lock' method.

Read-write lock:

Instead of locking the resource to both read and write operations equally. A read-write lock is distinguished into a two-part lock, i.e. a read-alone lock and a write-alone lock. By this way, it may improve the performance. By this way, it allows multiple threads to read the shared resources simultaneously, for reading doesn't cause data race. It allows a single thread to write on the shared resource at a moment. This type of lock may be used when the number of reading threads is much bigger than the number of writing thread.

Note: read-write lock may use more resource under the hood.

Liveness

Deadlock:

Resource A and its Lock A; Resource B and its Lock B. Thread A hold Lock A, while Thread b hold Lock B.  Thread A acquires Lock B, while thread B acquires Lock A. 

How to prevent threads from deadlock?
lock priority: multiple threads acquire locks in a fixed order.
lock timeout: if cannot acquire, then quit.

Abandoned lock: 

when one thread acquired the lock, but it fails to unlock it due to exceptions. The lock is therefore abandoned. This may lead to the rest of the threads in stuck.

How to prevent a lock becoming abandoned?

using try-catch block to surround the critical section, and in the 'finally' block unlock the lock.

Starvation:

Starvation happens when a thread is perpetually denied resources due to competition with others. A thread may in starvation because it has a low priority. Or it simply because there are too many concurrent threads.

Livelock: 

Multiple threads are designed to respond to each other so as to solve the conflict.

To solve the livelock, making sure only one thread takes action. It may take use of random selection.

Explicit Lock vs Implicit Lock

Explicit locks, such as those provided by the java.util.concurrent.locks package (e.g., ReentrantLock, ReadWriteLock), offer several advantages over using intrinsic locks provided by synchronized blocks or methods. Some of the key benefits of using explicit locks are:

1. Flexibility: Explicit locks offer more flexibility in terms of lock acquisition and release. You can lock and unlock the lock object at different scopes and in separate blocks of code, which is not possible with synchronized blocks or methods.

2. Fairness: ReentrantLock allows you to specify fairness when creating the lock object. When a lock is fair, it grants access to the longest-waiting thread. Intrinsic locks provided by synchronized blocks or methods do not have a fairness option and may lead to thread starvation in some scenarios.

java
Lock lock = new ReentrantLock(true); // Create a fair lock

3. Try to acquire lock: Explicit locks provide the tryLock() method, which allows a thread to attempt to acquire the lock without waiting indefinitely. This can help avoid deadlocks and improve performance in some cases.

java
if (lock.tryLock()) {
    try {
        // Perform the operation
    } finally {
        lock.unlock();
    }
} else {
    // Handle the lock not being acquired
}

4. Interruptible lock acquisition: With explicit locks, you can use the lockInterruptibly() method, which allows a thread to be interrupted while waiting to acquire the lock. Intrinsic locks do not support this functionality.

java
try {
    lock.lockInterruptibly();
    try {
        // Perform the operation
    } finally {
        lock.unlock();
    }
} catch (InterruptedException e) {
    // Handle the interruption
}

5. Separate read and write locks: The ReadWriteLock interface provides separate locks for read and write operations, allowing multiple threads to read shared data simultaneously while still providing exclusive access for write operations. This can lead to better performance in scenarios with more reads than writes.

java
ReadWriteLock rwLock = new ReentrantReadWriteLock();
Lock readLock = rwLock.readLock();
Lock writeLock = rwLock.writeLock();

While explicit locks offer several advantages, they also require more careful handling and can be more error-prone due to the need for manual lock management. When choosing between explicit locks and intrinsic locks, consider the specific use case, the complexity of the code, and the required functionality.