Modeling and simulation of proton-exchange membrane fuel cells (PEMFC) may work as a powerful tool in the Research & development of renewable energy sources. The Open-Source PEMFC Simulation Tool (OPEM) is a modeling tool for evaluating the performance of proton exchange membrane fuel cells. This package is a combination of models (static/dynamic) that predict the optimum operating parameters of PEMFC. OPEM contained generic models that will accept as input, not only values of the operating variables such as anode and cathode feed gas, pressure and compositions, cell temperature and current density, but also cell parameters including the active area and membrane thickness. In addition, some of the different models of PEMFC that have been proposed in the OPEM, just focus on one particular FC stack, and some others take into account a part or all auxiliaries such as reformers. OPEM is a platform for collaborative development of PEMFC models.

This paper presents a per-frame image-to-image translation system enabling copying of a motion of a person from a source video to a target person. For example, a source video might be a professional dancer performing complicated moves, while the target person is you. By utilizing this approach, it is possible to generate a video of you dancing as a professional. Check the authors' [video](https://www.youtube.com/watch?v=PCBTZh41Ris) for the visual explanation.

**Data preparation**

The authors have manually recorded high-resolution video ( at 120fps ) of a person performing various random moves. The video is further decomposed to frames, and person's pose keypoints (body joints, hands, face) are extracted for each frame. These keypoints are further connected to form a person stick figure. In practice, pose estimation is performed by open source project [OpenPose](https://github.com/CMU-Perceptual-Computing-Lab/openpose).

**Training**
https://i.imgur.com/VZCXZMa.png

Once the data is prepared the training is performed in two stages:
1. **Training pix2pixHD model with temporal smoothing**.
    
   The core model is an original [pix2pixHD](https://tcwang0509.github.io/pix2pixHD/)[1] model with temporal smoothing. Specifically, if we were to use vanilla pix2pixHD, the input to the model would be a stick person image, and the target is the person's image corresponding to the pose. The network's objective would be $min_{G} (Loss1 + Loss2 + Loss3)$, where: 

 - $Loss1 = max_{D_1, D_2, D_3} \sum_{k=1,2,3} \alpha_{GAN}(G, D_k)$ is adverserial loss;
 
 - $Loss2 = \lambda_{FM} \sum_{k=1,2,3} \alpha_{FM}(G,D_k)$ is feature matching loss;

  - $Loss3 = \lambda_{VGG}\alpha_{VGG}(G(x),y)]$ is VGG perceptual loss.

 However, this objective does not account for the fact that we want to generate video composed of frames that are temporally coherent. The authors propose to ensure *temporal smoothing* between adjacent frames by including pose, corresponding image, and generated image from the previous step (zero image for the first frame) as shown in the figure below:

  https://i.imgur.com/0NSeBVt.png

   Since the generated output $G(x_t; G(x_{t-1}))$ at time step $t$ is now conditioned on the previously generated frame $G(x_{t-1})$ as well as current stick image $x_t$, better temporal consistency is ensured. Consequently, the discriminator is now trying to determine both correct generation, as well as temporal consitency for a fake sequence $[x_{t-1}; x_t; G(x_{t-1}), G(x_t)]$.

2. **Training FaceGAN model**.

   https://i.imgur.com/mV1xuMi.png
  
   In order to improve face generation, the authors propose to use specialized FaceGAN. In practice, this is another smaller pix2pixHD model (with a global generator only, instead of local+global) which is fed with a cropped face area of a stick image and cropped face area of a corresponding generated image (from previous step 1) and aims to generate a residual which is added to the previously generated full image. 

**Testing**

 During testing, we extract frames from the input video, obtain pose stick image for each frame, normalize the stick pose image and feed it to pix2pixHD (with temporal consistency) and, further, to FaceGAN to produce final generated image with improved face features. Normalization is needed to capture possible pose variation between a source and a target input video.

**Remarks**

While this method produces a visually appealing result, it is not perfect. The are several reasons for being so:

 1. *Quality of a pose stick image*: if the pose detector "misses" the keypoint, the generator might have difficulties to generate a properly rendered image;
 2. *Motion blur*: motion blur causes pose detector to miss keypoints;
 3. *Severe scale change*: if source person is very far, keypoint detector might fail to detect proper keypoints.

Among video rendering challenges, the authors mention self-occlusion, cloth texture generation, video jittering (training-test motion mismatch).

References:

[1] "High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs"

This paper proposes a new training method for multi-agent communication settings. They show the following referential game: A speaker sees an image of a 3d rendered object and describes it to a listener. The listener sees a different image and must decide if it is the same object as described by the speaker (has the same color and shape). The game can only be completed successfully if a communication protocol emerges that can express the color and shape the speaker sees.

The main contribution of the paper is the training algorithm. The speaker enumerates the message that would maximise its own understanding of the message given the image it sees (in a greedy way, symbol by symbol). The listener, given the image and the message, predicts a binary output and is trained using maximum likelihood given the correct answer. Only the listener is updating its parameters - therefore the speaker and listener change roles every number of rounds.

They show that a compositional communication protocol has emerged and evaluate it using zero-shot tests.

[Implemenation of this paper in pytorch](https://github.com/benbogin/obverter)

PyCM is a multi-class confusion matrix library written in Python that supports both input data vectors and direct matrix, and a proper tool for post-classification model evaluation that supports most classes and overall statistics parameters.	PyCM is the swiss-army knife of confusion matrices, targeted mainly at data scientists that need a broad array of metrics for predictive models and an accurate evaluation of large variety of classifiers.

Joint summary from https://mitpress.mit.edu/books/developmental-robotics

Developmental robotics is the interdisciplinary approach to the autonomous design of behavioural and cognitive capabilities in artificial agents (robots) that takes direct inspiration from the developmental principles and mechanisms observed in the natural cognitive systems.  It relies on a highly interdisciplinary effort of empirical developmental sciences such as developmental psychology, neuroscience, and comparative psychology, and computational and engineering disciplines such as robotics and artificial intelligence.  The implementation of these principles and mechanisms into a robot-s control architecture and the testing through experiments where the robot interacts with its physical and social environment simultaneously permits the validation of such principles and the actual design of complex behavioural and mental capabilities in robots. Developmental psychology and developmental robotics mutually benefit from such a combined effort.

Synonym terms of Developmental Robotics include cognitive developmental robotics, autonomous mental development as well as epigenetic robotics \cite{Cangelosi18}. The  later term borrows the term ‘epigenetic’ from Piaget’s Epigenetic Theory of human development, where the child’s cognitive system develops as a result of the interaction between genetic predispositions and the organism’s interaction with the environment.  Therefore, ‘Epigenetic robotics’ was justified by Piaget’s stress on the importance of the role of interaction with the environment (later complemented with the emphasis on social interaction of Zlatev and Balkenius in 2001), and in particular, on the sensorimotor bases of higher-order cognitive capabilities. 
Due to the challenges of robotics domain not only on acquisition of skills in an incremental manner but also in open-ended environments, it is perhaps worth noticing that the state of the art is not as advanced as in other more constrained well defined unique task settings. Because of the importance of embodiment in robotics for open ended learning \cite{Cangelosi18} and because of the scarcity of continual learning strategies on robotics and merely assessed  on perception tasks, we believe both fields methodologies should borrow from each other. The blend could result not only in more effective or robust lifelong learning but also in less forgetful agents (i.e., the main rationale behind CL), which has not yet empirically shown is impossible to reach. We hope this survey helps reach this goal.

Dynamical systems development: In math it is characterized by complex changes over time in the phase state. The complex interaction of nonlinear phenomena results in the production of unpredictable states of the system, often referred to as emergent states. This concept was borrowed by developmental psychologists \cite{Thelen94} to explain child development as the emergent product of the intricate and dynamic interaction of many decentralized and local interactions related to the child growing body and brain and her environment.


Some definitions: 

Novelty, curiosity and surprise: focuses on the autonomy of learning, i.e. on the agents freedom to choose what, when and how it will learn. Intrinsic motivation (IM) is a mechanism to drive autonomous learning, not only in developmental robotics but also more broadly within the field of machine learning \cite{Mirolli13}\cite{Oudeyer07}. IM is task-independent, the agent could be placed in a completely new environment with no prior knowledge  or experience, and through self-directed exploration the robot will potentially learn not only important features of the environment but also the behavioural skills necessary for dealing with the environment. Second, IM promotes the hierarchical learning rather than solving a specific, predefined task. 

The Theory of MInd (ToM): A robot can use its own theory of mind to improve the interaction with human users, for example. It aims at reading the intention of the others and reacting appropriately to the emotional, attentional and cognitive states of the other agents, to anticipate their reactions, and to modify its own behaviour to satisfy these expectation and needs \cite{Scasellati02}.

Cascades: developmental theories refer to the far reach of early developments on later ones in terms of the “developmental cascade”.

Saliency in robotics: 
In robotics vision, a set of feature extraction methods typically can be applied to derive a saliency map \cite{Itti01}, i.e. the identification of the parts of the image that are important for the robot behaviour. It can be created, e.g., by combining a set of feature extraction method for color, motion, orientation and brightness. These features are combined to generate the whole saliency  map used by the model to fixate the objects.