Fraser plays Tetris is a reinforcement learning project based on python libraries PyGame and PyTorch. The algorithm is based on a project by patrickloeber on GitHub which revolved around the classic game Snake. I modified the code to work for Tetris, here are some of my findings.

Find my code here.

State

In reinforcement learning, the state serves as the foundation for decision-making, providing the agent with information about the environment. The state representation determines what aspects of the environment are observable to the agent and influences its ability to perceive, learn, and generalize across different situations. A well-designed state representation should capture relevant features of the environment while minimizing dimensionality and computational complexity. For my algorithm I studied 4 different states.

1. Full Information State Representation: In the full information approach, the state representation includes a binary grid indicating the occupancy of each cell on the Tetris board. It also includes information about the position, shape, and orientation of the falling Tetris block. This representation provides complete information about the current game state but may suffer from high dimensionality and scalability issues.
2. Plane Information State Representation: The plane information approach simplifies the state representation by providing information about the highest occupied plane in the placed blocks and the lowest occupied plane in the falling Tetris block. This representation aims to capture complementary patterns between the two planes to facilitate decision-making. It scales linearly with the width of the Tetris board.
3. Local Information State Representation: In the local information approach, the state representation focuses on the distance between each point in the falling Tetris block and the bottom floor of the board. It includes the number of points with the same distance and the shortest distance for each possible move. This representation allows the agent to have knowledge of the local space but limits its ability to plan multiple moves ahead. It is independent of the board size.
4. Minimal Information State Representation: The minimal information approach is a simplification of the local information method. It calculates the difference in the number of points with the same distance between the current state and the new state resulting from each possible action. The state consists of four values, one for each possible action, indicating the change in the number of points with the same distance. This representation aims to capture the immediate impact of each action on the local space.

State

The different states were evaluated by looking at the average score after the learning process has concluded. The average score is affected by model parameters, for example

• Learning Rate (Alpha): The learning rate, denoted as alpha, is a hyperparameter that controls the magnitude of updates to the model's parameters during training. It determines the size of the steps the algorithm takes when exploring the parameter space. A small learning rate may cause slow convergence and risk getting trapped in local minima, while a large learning rate may lead to unstable training and overshooting the optimal solution.
• Discount Factor (Gamma): The discount factor, denoted as gamma, is specific to reinforcement learning and influences the model's prioritization of long-term rewards versus immediate rewards. Gamma values range between 0 and 1, with higher values indicating a greater emphasis on long-term strategy.
• Exploration vs. Exploitation: During the initial stages of training, the algorithm employs exploration to randomly explore the state space and avoid local minima. Exploration allows the algorithm to discover diverse strategies and learn more about the environment. As the algorithm learns, it transitions towards exploitation, where it leverages the learned knowledge to make more informed decisions and exploit the most promising actions. Increasing the exploration step early in training can facilitate discovering optimal strategies and prevent premature convergence to suboptimal solutions.

In the plot below the optimization process is shown for state number 3 (Local Information State Representation) converging into a learning rate of 0.001 and a discount factor of 0.9. With these values the model reached an average of 2.0 points per game and a record of 14 points.

As a longterm music fan I have always been interested in what makes a song a specific genre. As I was studying the application of ML (Machine Learning) on audio signals for my work, I decided to use some of the methods I had learned in a side project. My final aim is to understand whether it is possible for a machine to descern a user's taste using ML.

I decided to begin with genre prediction and found an open dataset with 400 songs from 4 different genres which I used as my dataset and chose TensorFlow as my ML library. I initially attempted to feed waveform data directly into the algorithm. However, I encountered significant challenges due to slow performance and limitations in computational resources. Despite several attempts to optimize the algorithm, the computing power of my laptop proved insufficient to handle the complexity of the waveform data, hindering the training process.

I decided to take a different approach, and my research into the subject brought me to MFCC (Mel-Frequency Cepstral Coefficients) analysis, a technique widely utilized in voice recognition and related tasks. Thanks to this change I was able to overcome my performanxe issues and the training process sped up significantly. I then proceeded to optimize my algorithm by carefully selecting parameters such as model layers, layer depth and learning rate. My final algorithm achieved a validation accuracy of 90% and ROI of 0.78.

For future implementations of this algorithm I want to combine the two approaches. I will segment the original waveform in fractions that I will then use for the MFCC analysis, providing a change in frequencies over time.

Github repository.

Mycelium is a cozy puzzle game that asks you to play as the Mycelium Network connecting all trees within the forest. The aim of the game is to connect all trees using the least amount of resources as possible.

Connections

The connections are procedurally generated to branch out randomly every time. The code to genrate the connections was first written in python and then transferred to GDscript. It connects 2 points with many small segments all with a small random rotation. Once the connection has reached it's destination the trees are considered to be connected and the connection starts branching out, this occurs with passes where the in each pass the segrments from the previous passes branch out into new segments. Segment size, angle, and branching factor are randomly chosen for each connection.

Level design

The level is designed to be a slightly modified travelling salesmen problem, i.e. quickest path that connects all points with some paths being blocked by rocks. In later levels there will be two types of tree object, regular tree and resource tree. The resource trees will allow to increase the amount of resource in order to allow for more connections. In the image these are indicated by the trunks.

Assets

Tree asset art taken from Chuck CG.

Forest sounds from Mixkit.

Sound effects by Jordan Payne.

GDscript code found in GitHub repository.

During my doctoral research in quantum physics, I encountered a bottleneck in the data analysis process. The Python library I initially employed for fitting data to complex systems proved to be inefficient, particularly for the large datasets inherent in quantum experiments.

Seeking a solution, I embarked on a collaboration and ventured into learning Julia, a programming language known for its efficiency in scientific computing. Julia's optimization capabilities and efficient linear algebra operations offered a significant improvement over the Python library. The speed at which Julia processed and fitted the data was several orders of magnitude faster, enabling rapid extraction of insights from extensive datasets. As a result, I uncovered nuanced characteristics of the system that were previously obscured.

This experience underscored the value of collaboration and the importance of exploring alternative tools in scientific research. Learning Julia for this project was instrumental in accelerating the data analysis process and revealing novel insights into the physics of the system.

Resulting publication.

This project was a collaboration with a biologist, the primary objective was to provide quantitative data analysis for her study. The dataset consisted of microscopy videos capturing the post-fertilization movement of cortical fluids in ascidian cells, exhibiting different fluorescence frequency levels. My task involved extracting key metrics such as the cell's radius of curvature, fluorescence intensity distribution, and fluid speed from the 2D images.

To analyze the data, I utilized Python programming, developing algorithms for image processing. The process involved fitting the data to a custom function to extract cell walls, followed by calculating essential parameters like curvature and speed. This approach enabled us to produce quantitative insights into the biological processes under study.

The collaboration underscored the significance of interdisciplinary cooperation and was an enriching experience. Our findings were documented in a published paper, highlighting the value of combining expertise from different fields to advance scientific research.

Resulting publication.

My PhD work was concerned with two fascinating circuit quantum electrodynamics components, the Josephson junction and the geometric superinductor, and the interesting experiments that can be done by combining the two. The Josephson junction has revolutionized the field of superconducting circuits as a non-linear dissipation-less circuit element and is used in almost all superconducting qubit implementations since the 90s. On the other hand, the superinductor is a relatively new circuit element introduced as a key component of the fluxonium qubit in 2009. This is an inductor with characteristic impedance larger than the resistance quantum and self-resonance frequency in the GHz regime. The combination of these two elements can occur in two fundamental ways, in parallel and in series. When connected in parallel the two create the fluxonium qubit, a loop with large inductance and a rich energy spectrum reliant on quantum tunneling. On the other hand placing the two elements in series aids with the measurement of the IV curve of a single Josephson junction in a high impedance environment. In this limit theory predicts that the junction will behave as its dual element the phase-slip junction. While the Josephson junction acts as a non-linear inductor the phase-slip junction has the behavior of a non-linear capacitance and can be used to measure new Josephson junction phenomena, namely Coulomb blockade of Cooper pairs and phase-locked Bloch oscillations. The latter experiment allows for a direct link between frequency and current which is an elusive connection in quantum metrology. My work introduced the geometric superinductor, a superconducting circuit element where the high inductance is due to the geometry rather than the material properties of the superconductor, realized from a highly miniaturized superconducting planar coil. In my thesis these structures are be described and characterized as resonators and qubit inductors and progress towards the measurement of phase-locked Bloch oscillations is presented.

Thesis.

Disaster chemistry is an artistic porject inspired by the nanofabrication processes I was doing in my PhD. In particular I was fascinated by how mistakes in fabrication manifested as very interesting and abstract images. The art in disaster chemistry comes from overlaying and enhancing pictures taken with SEM and optical mictroscopes of samples that for one reason or another didn't qualify as being "useful" in the scientific sense. This project allowed me to process the frustration inherent to the trial and error nature of my work by helping me appreciate the value of mistakes and learning from them.

For more images visit my instagram.

One of my great passions is clothing and fashion. For years I have been designing and bringing to life unique garments for myself and friends. I really enjoy the design process such as choosing patterns and fabrics, and I find the process of cutting and sewing pieces together relaxing. Above you can find some of my creations.