The overarching goal of precision oncology is to use personal genomic information, like gene expression data from a tumor biopsy, to devise individualized treatment plans. Network-based analytics support this goal in three ways. First, by helping us to identify malfunctioning gene pathways by observing patient-specific gene expression levels in relation to one another. Second, by enabling us to predict the phenotype, or observable characteristics, of a patient's cancer. Third, by recommending drugs to re-purpose as chemotherapies that targeting a specific gene or protein that has been dysregulated in a pathway. 

## Identifying Pathological Gene Pathways
Cancer is best characterized as a disease caused by frequently mutated genes that cause dysregulated biological pathways. There are many kinds of biological pathways, such as metabolic pathways that may synthesize useful molecules or break down complex molecules, or a gene-regulatory pathway wherein molecules like proteins, mRNA, or DNA regulate the production, expression, and activity of other molecules thereby affecting the behaviour of the cell.

Systems biology makes use of many types of pathways, which may be viewed as molecular networks like the gene-regulatory network, the protein-protein interaction network, and the cellular signal transduction network. One goal of network-based analysis of personal genomic profiles is to identify network modules that are causative of cancer and informative of cancer phenotype. This paper outlines three ways of integrating molecular networks to make these predictions: model-based integration, pre-processing integration, and post-analysis integration.

The authors use the phrase **pre-processing integration** to indicate a two-step process. The first step detects molecular sub-networks by using highly discriminative genes as seed genes in a greedy search to find discriminative sub-networks. The gene expression in each sub-network is then normalized as one feature value for classification with logistic regression or another method.  This method allows you to customize your sub-network features, but generally does not produce optimal predictions.

An example application of this type of method comes from a paper called [Network-based classification of breast cancer metastasis](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2063581/). Their goal is to determine which gene pathways give rise to metastasis. They note that previous methods had identified certain genes as biomarkers correlated with metastasis. Their approach was to use these genes as seeds for an algorithm to identify discriminative sub-networks in a protein-protein interaction network.
https://i.imgur.com/WCf18fZ.png

To conduct the **post-analysis integration** of oncogenic alterations in networks, first analyze the patient genomic profiles to generate a list of oncogenic alterations. Second, map those alterations to the network as seed genes for the module analysis. This method is highly informative of cancer mechanisms in the network, but relies on accurate identification of oncogenic alterations in the patient data.
https://i.imgur.com/5CbqQTg.png

When using a **model-based integration** approach, you incorporate the molecular network into the machine learning model as a regularizing graph Laplacian. The coefficients that the model learns form dense sub-networks that are used to enable the model to predict patient survival or cancer phenotype. This form of framework has a global optimization strategy and generally produces the best outcome predictions.
https://i.imgur.com/vRH1K8R.png

## Methods for Drug Re-purposing
If you want to re-purpose a drug for use against a cancer target, there are three networks that are useful to inform your work: drug-drug similarity, drug-target relations, and gene-gene relations. Additionally, there are three types of methods that you can use with this data to create a predictive model: graph connectivity measures, link prediction models, and network based classification methods.

## Graph Connectivity Methods
Several studies have shown that drugs sharing similar chemical structures, side-effects, or similarities in gene-expression profiles following drug treatment, can be good candidates for re-purposing. It follows that we may combine a drug-drug interaction network, a drug-target interaction network, and a target-target interaction network to estimate the likelihood that a specific drug may be re-purposed for a specific target. The proxy that is used for similarity is simply length of the path from the query drug in the drug-drug interaction network to the target in the target-target interaction network.
https://i.imgur.com/abmHkKl.png

## Link Prediction Models
The link prediction model is close to the Graph Connectivity method, but relies on more advanced similarity measures and the global graph structure to determine relations between drugs and targets. For example, one paper using a link prediction model for drug repurposing uses drug-drug two dimensional structural similarity, target-target genomic sequence similarity, and a dataset of known drug-target interactions. Once these measures have been calculated two established methods are used to predict new drug-target relations: matrix completion and random walks.
https://i.imgur.com/kElewfE.png

## Network-based Classification Methods
Here we view drug repositioning as a classification problem and apply standard classification methods such as SVM. The inputs to the classifier are network topological features (local neighborhood) and the labels are from a dataset of drug-target relations describing whether a specific drug is effective against a specific target. You use the classifier to make predictions on the gene neighborhoods present in your test set.
https://i.imgur.com/cwfJr8t.png

## Network-Based Analysis of TCGA Mutation Data 
The paper also conducts a network-based analysis mutated genes in 31 cancer genome projects in TCGA using enriched KEGG pathways. The authors find that they are able to re-identify an important signalling pathway that requires the gene AMPK and is known to be affected in breast cancer (BRCA) and uterine corpus endometrial cancer (UCEC).

## Conclusion
This article surveys precision oncology, primarily discussing drug re-positioning and alorithms for integrating molecular networks with patient data. It also contains a case-study on ovarian cancer, and a network-based analysis of TCGA data using gene mutations.

## Future Work
The authors outline several directions for future work. One direction is to explore higher-resolution graphs that distinguish between isoforms of genes and capture more complex relationships between molecules. Another is to research heterogeneous cell populations in tumors using single-cell RNA sequencing to differentiate cells with different drug targets. Finally, it is essential to develop a standardized software platform that integrates biomedical, biological network data, and analytic software to support comprehensive network-based analysis of patient genomic data and drug re-positioning.

https://i.imgur.com/QxHktQC.png 
The fundamental question that the paper is going to answer is weather deep learning can be realized with other prediction model other thahttps://i.imgur.com/Wh6xAbP.pngn neural networks. The authors proposed deep forest, the realization of deep learning using random forest(gcForest). The idea is simple and was inspired by representation learning in deep neural networks which mostly relies on the layer-by-layer processing of raw features.

Importance: Deep Neural Network (DNN) has several draw backs. It needs a lot of data to train. It has many hyper-parameters to tune. Moreover, not everyone has access to GPUs to build and train them. Training DNN is mostly like an art instead of a scientific/engineering task. Finally, theoretical analysis of DNN is extremely difficult. The aim of the paper is to propose a model to address these issues and at the same time to achieve performance competitive to deep neural networks.

 Model: The proposed model consists of two parts. First part is a deep forest ensemble with a cascade structure similar to layer-by-layer architecture in DNN. Each level is an ensemble of random forest and to include diversity a combination of completely-random random forests and typical random forests are employed (number of trees in each forest is a hyper-parameter). The estimated class distribution, which is obtained by k-fold cv from forests, forms a class vector, which is then concatenated with the original feature vector to be input to the next level of cascade. Second part is a multi-grained scanning for representational learning where spatial and sequential relationships are captured using a sliding window scan (by applying various window sizes) on raw features, similar to the convolution and recurrent layers in DNN. Then, those features are passed to a completely random tree-forest and a typical random forest in order to generate transformed features. When transformed feature vectors are too long to be accommodated, feature sampling can be performed.

Benefits: gcForest has much fewer hyper-parameters than deep neural networks. The number of cascade levels can be adaptively determined such that the model complexity can be automatically set. If growing a new level does not improve the performance, the growth of the cascade terminates. Its performance is quite robust to hyper-parameter settings, such that in most cases and across different data from different domains, it is able to get excellent performance by using the default settings. gcForest achieves highly competitive performance to deep neural networks, whereas the training time cost of gcForest is smaller than that of DNN.

Experimental results: the authors compared the performance of gcForest and DNN by fixing an architecture for gcForest and testing various architectures for DNN, however assumed some fixed hyper-parameters for DNN such as activation and loss function, and dropout rate. They used MNIST (digit images recognition), ORL(face recognition), GTZAN(music classification ), sEMG (Hand Movement Recognition), IMDB (movie reviews sentiment analysis), and some low-dimensional datasets. The gcForest got the best results in these experiments and sometimes with significant differences.

My Opinions: The main goal of the paper is interesting; however one concern is the amount of efforts they put to find the best CNN network for the experiments as they also mentioned that finding a good configuration is an art instead of scientific work. For instance, they could use deep recurrent layers instead of MLP for the sentiment analysis dataset, which is typically a better option for this task. For the time complexity of the method, they only reported it for one experiment not all. More importantly, the result of CIFAR-10 in the supplementary materials shows a big gap between superior deep learning method result and gcForest result although the authors argued that gcForest can be tuned to get better result. gcForest was also compared to non-deep learning methods such as random forest and SVM which showed superior results. It was good to have the time complexity comparison for them as well. In my view, the paper is good as a starting point to answer to the original question, however, the proposed method and the experimental results are not convincing enough.

Github link: https://github.com/kingfengji/gcForest

[https://www.cs.uic.edu/~jakob/papers/biagioni-trr12.pdf](https://www.cs.uic.edu/~jakob/papers/biagioni-trr12.pdf) is a super nice survey where I almost feel bad to summarize it. Almost.

It is about road map inference in the presence of traces of geo coordinates. So you have many vehicles which log their positions while driving. From this data, you want to infer the latest map.

Problems:
* The map changes: What was a valid map a week ago, might not be anymore (due to construction work, breaking roads, new roads)
* Sensor errors: GPS is not very accurate

## Contributions

* overview over the literature up to 2012 on map generation
* method for the automatic evaluation of generated maps
* an evaluation of three reference algorithms including code
* a [118-h trace data set](https://www.cs.uic.edu/bin/view/Bits/Software) and ground truth map

## How Map Inference Works

* **Preprocess Geo-Traces**: check for unreasonable speed, too extreme acceleration, too abrupt changes
* Inference:
    * k-Means: Reduce candidates to centroid. Works on 3D - 2 coordinates and direction
    * trace merging: Merge edges directly, without reduction
    * kernel density estimation

## Quantitative Evaluation

1. Start at one location in both, the ground truth map and the generated map
2. Follow streets from both and sample points in a fixed range (e.g. each 50cm). Only go in directions that go away from the start
3. Try to match both, ground truth and generated map points. Only match if points are close enough (e.g. a distance of 10cm if this is an acceptable uncertainty). Then use a classification score (e.g. F1)

A high definition (HD) map in the context of highly automated vehicles is a digital map which contains precise information (error < 1m) necessary for localization and for driving behavior:

* **road geometry**: Where is the road, where are lanes?
* **[street furniture](https://en.wikipedia.org/wiki/Street_furniture)**: street signs, traffic lights
* **dynamic data**: end of traffic jam, construction work

This kind of information can be stored in the [OpenDRIVE format](http://www.opendrive.org/docs/OpenDRIVEFormatSpecRev1.1D.pdf), which is an XML format with the extension `.xodr`.

Creating and maintaining those maps is costly with dedicated cars, so we would like to have an alternative which works automatically. This paper proposes one. **The best summary of the paper is Figure 3.**

## Possible Sensors

The following kind of sensors can be equipped in many cars:

* stereo cameras
* radar
* GPS

## Active Players

* **Continental** Road Database: Automatic Recording and Processing of Highly Accurate Route Data
* [HERE](https://www.here.com/en): map provider which has published a sensor interface specification: “Vehicle Sensor Data Cloud Ingestion Interface
Specification (v2.0.2)

## See also

* [Inferring road maps from global positioning system traces - Survey and comparative evaluation](https://www.cs.uic.edu/~jakob/papers/biagioni-trr12.pdf)
* [Hochgenaue Fahrzeugeigenlokalisierung und kollektives Erlernen hochgenauer digitaler Karten]()

As in Q-learning, modern actor-critic methods suffer from value estimation errors due to high bias and variance. While there are many attempts to address this in Q-learning (such as Double DQN), not much was done in actor-critic methods.

Authors of the paper propose three modifications to DDPG and empirically show that they help address both bias and variance issues:

* 1.) Clipped Double Q-Learning:  
  Add a second pair of critics $Q_{\theta}$ and $Q_{\theta_\text{target}}$ (so four critics total) and use them to upper-bound the value estimate target update: $y = r + \gamma \min\limits_{i=1,2} Q_{\theta_{target,i}}(s', \pi_{\phi_1}(s'))$
* 2.) Reduce number of policy and target networks updates, and magnitude of target networks updates: $\theta_{target} \leftarrow \tau\theta + (1-\tau)\theta_{target}$
* 3.) Inject (clipped) random noise to the target policy: $\hat{a} \leftarrow \pi_{\phi_{target}}(s) + \text{clip}(N(0,\sigma), -c, c)$

Implementing these results, authors show significant improvements on seven continuous control tasks, beating not only reference DDPG algorithm, but also PPO, TRPO and ACKTR.

Full algorithm from the paper:

https://i.imgur.com/rRjwDyT.png

Source code: https://github.com/sfujim/TD3