Wednesday, October 23, 2024

PySerial talking to an Arduino can be slow. Putting an http server in between is one solution. Also which language? Yeah whatever.

This week I discovered how unacceptably slow Python can be when talking to an Arduino via PySerial.  I have an arduino controller driving some stepper motors, and it won't answer "OK" until the steppers have finished executing the requested movement.  Python introduced an extra 5-6 seconds per move instruction, when compared to issuing the same move commands from a TeraTerm script doing the same thing.  So there wasn't anything wrong with the Arduino controller - all the blame sat on the Python side.  I tried several suggestions I found online and none of them got me any closer.  It's wierd because if I modded the Arduino controller to just be an echo of any incoming characters, then Python was instanteous in handling the responses and going to the next command.

To work around this, I set up an http server (specifically with Go - and I only chose that because I was hitting my head when I tried to include Crow in C++, and I really don't care which language the server is in, all I cared about was verifying that Python would work better and as fast as I possibly could verify that, so, sorry C++, maybe next time).

This is an interesting side-note.  Languages don't mean what they used to, especially when one is brand new to me and the other I haven't used in quite a while, or never for some application.  For a simple little http server talking to a serial port, the only thing I cared about was "how long does it take me to get to a running proto when I am practically a newb in both languages for this application?"  And in this case, there was too much tribal knowledge assumed by the C++ camp, while Go seemed to have a shorter learning curve.  This might be completely unfair for me to put into writing, because the whole time I was following my gut - but the point is, you might be better off going into a new language if it seems to have a lot of support.  All of the languages sort of blur together with Copilot or ChatGPT writing **something** to get me started.

Okay for a little detail, the server expects routes that will be the commands sent to the Arduino stepper controller.  For example a server route of http://localhost/movr?x=10&y=20&z=30 would result in sending the Arduino a movr,10,20,30 command, waiting for it to send back OK, and then reply to the Python script that browsed to that address with an OK, and Python could move to the next movr command - and this worked out great.  Bye bye pyserial!

Monday, November 27, 2023

Running a PS3 emulator (and could this be done in Unity?)



Saturday I picked up a used GT4 on craigslist. There are several Playstation emulators that allow you run this on a PC. Sony publishes the bios/firmware for PS3 on their website, and there's a tool that extracts the PS1 and PS2 firmware from that file, but the tool isn't from Sony - some third party posted it (or more likely it is from Sony, but it's just not supported). That was the only "risky" thing involved in the process. It's very impressive to me that the PS1 and PS2 firmwares are somehow embedded into the PS3 firmware, accessible via an extraction tool.


I also picked up GT5 but that requires a Blu-ray reader, which is on order. GT4 plays almost exactly like the original, but there are some quirks. It's essentially a virtual machine of a PS3 (or rather PS2) and some features don't always work, or they work only with a certain setting turned on and that same setting crashes if it's on at the wrong time, so just live without it.

This whole exercise made me wonder what sort of effort it would be to write a virtual machine player in C#, and import that into Unity, and write a game that has a gaming console in the game so you can play a game within a game. :)

And the answer is, it's as easy as following this nice little article, and screaming in horror at the sight of ASM. But I'm sure it would be a walk in the park...
https://www.codeproject.com/KB/recipes/B32Machine1/VMCS.pdf

Saturday, December 31, 2022

ChatGPT Unity Graph Database Code Example

UPDATE: This code eventually worked, although I had to ask ChatGPT to also write the PriorityQueue class which it had originally left undeclared (I didn't include this part of the chat below - but reproducing what I ran shouldn't be too difficult for anyone who is interested in retracing my steps).  Running this code also ran into an exception in the sort function, which could be fixed by expanding the lambda function (I don't like those much anyways) into a full function which tested that both sort keys exist and if they don't just return 0 in the (formerly) lambda function for the sort.  Anyways... it all works as expected and helped me go forward with this coding idea in Unity.  If you want the full details, please let me know and I'll be happy to add them here.


 


https://chat.openai.com/chat

Frank Braker
write a graph database for the unity3d environment to be used with wavefunction collapse map generation and path finding

ChatGPT:

A graph database is a database that stores data in a graph structure, with nodes representing entities and edges representing relationships between those entities. In the context of the Unity3D environment, you might use a graph database to store information about the layout of a game level, such as the locations of objects, obstacles, and points of interest. This information could then be used to generate maps using wavefunction collapse algorithms and to perform pathfinding operations to navigate through the game level.

To create a graph database for use in Unity3D, you would first need to design the data model for your graph. This would involve determining the types of nodes and edges you will need to represent the data you want to store, and defining any properties or attributes that each node or edge might have.

Next, you would need to implement a data structure to store your graph in memory. There are several options you could consider for this, including adjacency lists, adjacency matrices, and edge lists. The choice of data structure will depend on the specific requirements of your application, including the size and complexity of the graph and the type of operations you will be performing on it.

Once you have implemented the data structure for your graph, you can then create functions to add and remove nodes and edges, as well as to query and manipulate the data stored in the graph. You may also want to consider implementing algorithms for wavefunction collapse map generation and pathfinding, such as Dijkstra's algorithm or A* search.

Finally, you would need to integrate your graph database into your Unity3D project, either by writing code to access the database from within your game or by using an existing plugin or library that provides this functionality. This would allow you to use the data stored in the graph to generate maps and perform pathfinding in your game.


Frank Braker
write the C# code for this

ChatGPT:
Here is an example of how you could implement a basic graph database in C# for use in the Unity3D environment:

using System;

using System.Collections.Generic;

 

// A node in the graph

public class Node {

  // The unique ID of the node

  public int ID { get; set; }

  // A dictionary of edges connecting this node to other nodes

  public Dictionary<int, Edge> Edges { get; set; }

  // Any additional properties or attributes of the node

  public object Data { get; set; }

 

  public Node(int id) {

    ID = id;

    Edges = new Dictionary<int, Edge>();

  }

}

 

// An edge in the graph, connecting two nodes

public class Edge {

  // The unique ID of the edge

  public int ID { get; set; }

  // The node at the start of the edge

  public Node Start { get; set; }

  // The node at the end of the edge

  public Node End { get; set; }

  // The weight of the edge, if applicable

  public int Weight { get; set; }

  // Any additional properties or attributes of the edge

  public object Data { get; set; }

 

  public Edge(int id, Node start, Node end, int weight = 0) {

    ID = id;

    Start = start;

    End = end;

    Weight = weight;

  }

}

 

// The graph itself

public class Graph {

  // A dictionary of all the nodes in the graph

  public Dictionary<int, Node> Nodes { get; set; }

  // A dictionary of all the edges in the graph

  public Dictionary<int, Edge> Edges { get; set; }

 

  public Graph() {

    Nodes = new Dictionary<int, Node>();

    Edges = new Dictionary<int, Edge>();

  }

 

  // Add a new node to the graph

  public void AddNode(int id, object data = null) {

    Nodes[id] = new Node(id) { Data = data };

  }

 

  // Add a new edge to the graph

  public void AddEdge(int id, Node start, Node end, int weight = 0, object data = null) {

    var edge = new Edge(id, start, end, weight) { Data = data };

    start.Edges[id] = edge;

    Edges[id] = edge;

  }

 

  // Remove a node and all its connected edges from the graph

  public void RemoveNode(int id) {

    var node = Nodes[id];

    foreach (var edge in node.Edges.Values) {

      Edges.Remove(edge.ID);

    }

    Nodes.Remove(id);

  }

 

  // Remove an edge from the graph

  public void RemoveEdge(int id) {

    var edge = Edges[id];

    edge.Start.Edges.Remove(id);

    Edges.Remove(id);

  }

}

This code defines a Node class to represent a node in the graph, an Edge class to represent an edge connecting two nodes, and a Graph class to represent the graph itself. The Graph class has functions to add and remove nodes and edges, as well as a

EDIT: Here ChatGPT never finished its sentence, but that's fine.

Frank Braker
write a pathfinding function to find the fastest path between 2 nodes, which returns a list of all intermediate nodes take to form the path

ChatGPT:
Here is an example of a pathfinding function that you could use to find the fastest path between two nodes in a graph, using the Dijkstra algorithm:

using System;

using System.Collections.Generic;

 

// Find the fastest path between two nodes in the graph using Dijkstra's algorithm

public List<Node> FindFastestPath(Graph graph, Node start, Node end) {

  // Set up a priority queue to store the nodes we need to visit

  var queue = new PriorityQueue<Node>();

 

  // Set up a dictionary to store the distance from the start node to each node

  var distances = new Dictionary<Node, int>();

  // Set the distance to the start node to 0, and all other nodes to infinity

  distances[start] = 0;

  foreach (var node in graph.Nodes.Values) {

    if (node != start) {

      distances[node] = int.MaxValue;

    }

  }

 

  // Set up a dictionary to store the previous node on the fastest path from the start node to each node

  var previous = new Dictionary<Node, Node>();

 

  // Add the start node to the queue

  queue.Enqueue(start, 0);

 

  // While there are still nodes in the queue

  while (queue.Count > 0) {

    // Dequeue the node with the lowest distance

    var current = queue.Dequeue();

 

    // If we have reached the end node, we can stop the search

    if (current == end) {

      break;

    }

 

    // For each neighbor of the current node

    foreach (var neighbor in current.Edges.Values) {

      // Calculate the distance from the start node to the neighbor through the current node

      var distance = distances[current] + neighbor.Weight;

 

      // If the distance from the start node to the neighbor through the current node is shorter than the current distance from the start node to the neighbor

      if (distance < distances[neighbor.End]) {

        // Update the distance from the start node to the neighbor

        distances[neighbor.End] = distance;

        // Update the previous node on the fastest path from the start node to the neighbor

        previous[neighbor.End] = current;

        // Add the neighbor to the queue with the updated distance

        queue.Enqueue(neighbor.End, distance);

      }

    }

  }

 

  // If we have a path from the start node to the end node

  if (previous.ContainsKey(end)) {

    // Create a list to store the nodes on the fastest path

    var path = new List<Node>();

    // Set the current node to the end node

    var current = end;

    // While we have not reached the start node

    while (current != start) {

      // Add the current node to the beginning of the list

      path.Insert(0, current);

      // Set the current node to the previous node on the path

      current = previous[current];

    }

    // Add the start node to the beginning of the list

    path.Insert(0, start);

    // Return the list of nodes on the fastest path

    return path;

  }

  // If there is no path from the start node to the end node, return an empty list

  return new List<Node>();

}


Frank Braker
write example C# code that creates a map comprised of several nodes and edges, and which uses the pathfinding function, using the graph database code and pathfinding above

ChatGPT:

Here is an example of C# code that creates a map comprised of several nodes and edges, and uses the pathfinding function to find the fastest path between two nodes:


using System;

using System.Collections.Generic;

 

public static void Main() {

  // Create a new graph

  var graph = new Graph();

 

  // Add some nodes to the graph

  graph.AddNode(0, "Node 0");

  graph.AddNode(1, "Node 1");

  graph.AddNode(2, "Node 2");

  graph.AddNode(3, "Node 3");

  graph.AddNode(4, "Node 4");

  graph.AddNode(5, "Node 5");

 

  // Add some edges to the graph, with weights indicating the cost to travel along the edge

  graph.AddEdge(0, graph.Nodes[0], graph.Nodes[1], 10);

  graph.AddEdge(1, graph.Nodes[0], graph.Nodes[2], 5);

  graph.AddEdge(2, graph.Nodes[1], graph.Nodes[3], 1);

  graph.AddEdge(3, graph.Nodes[1], graph.Nodes[4], 2);

  graph.AddEdge(4, graph.Nodes[2], graph.Nodes[1], 3);

  graph.AddEdge(5, graph.Nodes[2], graph.Nodes[4], 9);

  graph.AddEdge(6, graph.Nodes[3], graph.Nodes[2], 2);

  graph.AddEdge(7, graph.Nodes[3], graph.Nodes[5], 4);

  graph.AddEdge(8, graph.Nodes[4], graph.Nodes[3], 6);

  graph.AddEdge(9, graph.Nodes[4], graph.Nodes[5], 2);

 

  // Find the fastest path from node 0 to node 5

  var path = FindFastestPath(graph, graph.Nodes[0], graph.Nodes[5]);

 

  // Print the IDs of the nodes on the fastest path

  Console.WriteLine("Fastest path from node 0 to node 5:");

  foreach (var node in path) {

    Console.WriteLine(node.ID);

  }

}

This code creates a new graph and adds six nodes and ten edges to it. It then uses the FindFastestPath function to find the fastest path from node 0 to node 5, and prints the IDs of the nodes on the path to the console. The output of this code would be:

Fastest path from node 0 to node 5:

0

2

1

3

5





Tuesday, June 14, 2022

Optimal Buy and Sell Estimation Using BetterInvesting.com

Optimal Buy and Sell Estimation Using BetterInvesting.com

Authored by: Frank Braker


Disclaimer: None of the words, thoughts, concepts, images or ideas in this post should be construed as financial advice of any kind.  This is a discussion of the features of a tool and not a recommendation of any financial strategy.  If you trade stocks, you assume all trading risks yourself.

I recently started using BetterInvesting.com.  In this guide you will learn about tools found there.


What The Tools Provide

These tools allow answering the following questions:

  • Is a company healthy?

    In the vast ocean of publicly tradable companies, we find a spectrum of varying degrees of company health, from the highly profitable to those that will soon declare bankruptcy.  Within seconds any given company can be visually understood in terms of Sales relative to competitors, Sales growth, Earnings per Share, Stock price, Pre-tax profit on Sales, ROE (return on equity), Total Debt, and Debt to Capital.  These indicators are presented visually to allow seeing long-term trends.

  • Should I buy this company now?  Sell it?  How can I tell?

    Forecasting high and low estimated prices based on historical data and fundamentals allows a methodical process for determining where to buy and where to sell - no more guessing or hoping for a price that may never happen.


2 stock research tools are provided at BetterInvesting.com:

  • CoreSSG: this is “the tutorial” mode of the toolset.  Use this for guidance and a detailed explanation of each step in evaluating a company.  A limited number of stock symbols can be fully evaluated for free in the trial version of BetterInvesting.com

  • SSGPlus: this is “the advanced” mode of the toolset which offers a condensed overview in a few screens.


Price

To complete this guide, you won’t have to pay anything.  When you are finished, consider buying the tools to open an account at Betterinvesting.com.  The price is roughly $10/month for the full suite of tools, and a marginally cheaper price for the “basic” tools.  I recommend buying the full suite for a full year as it is relatively affordable, especially for the service the tools provide.

Workflow

If you go to BetterInvesting.com - there’s a lot of interesting fluff to read.  You can join groups and spend time reading the site wondering what all this is.

Instead of wondering what this is, open this link instead:
https://ssg.betterinvesting.org/trial/ssgplus.aspx?ticker=fds

Notice that I have highlighted “ssgplus” in the link.  We’ll see why, below, but first have a look at the screen the link gives - it will look something like this:


This is the advanced tool.  For a summary screen of determining a company’s health, make sure the green “sales”, pink “pre-tax profit” and the blue “EPS (earnings per share)” are all sloping upwards from left to right.  Other criteria under “Legend” on the right can also be enabled.

For a deeper understanding of the meaning of the summary screen above, let’s switch to the CoreSSG tool by opening the URL below (this is the same URL as above, only ‘ssgplus’ is changed to ‘coressg’):
https://ssg.betterinvesting.org/trial/coressg.aspx?studyid=14852657&sk=48125


You could change which stock symbol you are looking at by clicking on “Quick Navigation”, but in the trial version only a few different symbols will be available.  Let’s discuss a little about how to find good stocks to analyze in the next section.

Finding Good Stocks

The full product does have a ‘heat map’ of studies other BetterInvesting users have done, which is a good method for finding stocks to look at.  There are more brute force methods such as using scripts to open the website in a browser to quickly navigate many symbols, see at a glance whether they are interesting, and look at hundreds of symbols in one sitting to try to find “hidden gems”.  This method won’t be discussed here, and is of nominal advantage over using the heat map (as companies that aren’t generating a lot of news buzz won’t see a lot of market volume either).

Using The CoreSSG Tool

The screen you brought up (here’s the link again) should look as shown in the screenshot below
https://ssg.betterinvesting.org/trial/coressg.aspx?studyid=14852657&sk=48125 :




Because CoreSSG is like a “tutorial mode” - it provides “Back” and “Next buttons to go through all the steps for evaluating a stock (just under the “Analyze Growth” button shown below:

You can study the notes given in the “Education” field which are very helpful:


Here are screenshots for each step of CoreSSG.  Again, the Education field provides good background for what is shown in each screenshot.  There will be some additional discussion provided below.

Figure 1: Historical Sales


Figure 2: Historical Earnings Per Share

Figure 3: Historical Stock Price Review


Figure 4: % Pre-Tax Profit on Sales


Figure 5: % Return on Equity


Figure 6: Total Debt


Figure 7: % Debt to Capital


Figure 8: Forecast Sales


Figure 8: Forecast Sales is the first screen in CoreSSG which requires filling in the “Enter Your 5 Year Annual Sales Growth Rate Forecast (%)” field.  Try using the same value as shown under “Growth Rate(%)” in the lower left corner of the screen, or adjust your estimate according to the current economic situation (i.e. is the economy booming, or going into a recession, etc.).  There are other details provided in the “Education” field too.

Figure 9: Forecast Earnings


Similarly, Figure 9: Forecast Earnings asks that you fill in the EPS Growth Rate Forecast (%) field.

By filling in these forecasts of Sales and Earnings, high and low estimates of PE and EPS give a basis for the high and low price estimates which will establish when to buy and when to sell, based on these forecasts.  We’ll see this in later steps below.


Figure 10: Forecast High Price

The forecast high price is calculated from your average high PE forecast multiplied by your high EPS forecast.  In Figure 10: Forecast High Price, values of 30 and 15 were entered for MSFT, which will be different for whatever symbol is being evaluated.  After entering these values, be sure to click “Update Study”.


Discussion

Forecasting the future PE, in practice, simply means putting in a reasonable number based on the 2 average P/E values shown in the lower left of the screen (just above the “Update Study” button).  Find the EPS forecast of “Figure 9: Forecast Earnings”, marked as “Analyst Consensus Estimate: EPS Long Term Estimate” shown in the figure as 13.8%.

It is worth noting: the more “glamorous” a stock is, the more “enthusiastic” (or perhaps “insane”) the market tends to be.  The market will pay less attention to company fundamentals and more on hype.  So as a result, the PE forecast will be higher than a more glamorous stock.  I.e. the price will be higher, regardless of earnings, for such a stock.

You will probably start to notice that, when comparing stocks and all other things being equal (for example earnings), the price will be inflated or deflated relative to the “glamor” of a stock.  For a less popular stock with excellent fundamentals, the PE will tend to be lower than if it is a well-known brand name with the same fundamentals or worse.  You will start to see a range of low PE which marks a good support price (where investors will buy the stock, causing it to go up in price, when it had previously been falling), and that range of low PE will be specific to that symbol, according to how glamorous that stock is.  NOTE: It begs the question, do some companies inflate their brand-name even when they have poor fundamentals, in order to inflate stock price?  By using these tools, we can avoid such companies by getting a clear picture of their fundamentals (both current and historical).


Figure 11: Forecast Low Price


For Figure 11: Forecast Low Price, follow a similar process as you did for Figure 10: Forecast High Price.


Figure 12: Assess Stock Price


Figure 12: Asses Stock Price provides a range of prices that indicate where to place a buy order, and where to place a sell order, using your previous estimates.  It will show “Invalid” if there are any issues with the estimates you provided.


Figure 13: Determine Potential Gain vs. Loss

In Figure 13: Determine Potential Gain vs Loss, the “Upside Downside Ratio” gives the probably of gain versus loss - and is recommended that at least 3 times the probability of gain makes an investment desirable.

Figure 14: Determine 5 Year Potential


This figure shows the annual rate of return assuming you are able to trade at the forecast high and low P/E.


Figure 15: Notes


Staff, community and your own notes can be accessed here.



Example Analysis
Here is an example. To generate this, I brought up a yearly chart of SWKS from 2017 to 2021, with the PE ratio as a charted indicator, and could from that, year by year, determine the yearly high and yearly low PE ratio. I could that way generate the 5 year averages by hand. But because the current year's low PE is far below this average, I had to skew both the high and low averages to a possible new low PE this year (possibly even lower than what we have right now). You should see those adjusted high and low PE estimates presented here:




Summary

By using the CoreSSG or SSGPlus tools, you can remove the guesswork of how healthy a company is, and calculate buy and sell prices for trading that company. Compared to technical trading, this is a very slow strategy that depends on yearly cycles, rather than short-term news events, to drive when to get in and out of a position.



Friday, March 18, 2016

Inheritance versus ECS Pattern Concerns

"Should these data members become the base class that this class inherits from?"  If you've ever heard this phrase, or had any discussion involving multiple inheritance, read on!

If you've read anything about the Entity Component System design pattern (ECS), you might have dismissed it as being only relevant to game design.  But ECS short-circuits the need for inheritance - so it's worth trying to understand as a tool in your belt.

I started writing this with processing architectures in mind, because it helps to visualize how the ECS pattern seems to work.  So I will first enumerate computer processing architectures I have encountered.


From The Main Event Loop to Multi-Threaded OS

If you go back to before operating systems offered multi-tasking, code was written in a loop which waited for events, and every event was then processed.  There were several weaknesses - but we don't care for this article.

Later came multi-tasked operating systems.  They allowed events to be queued up in one (or several) tasks.  Other tasks could process event handling.  Then finally we have multi-threaded operating systems.  As more processors are available, tasks can be assigned to other processors.


But all we really need to understand is the main event loop, as it mimics the processing needed to make ECS work on a per-object basis.


The Entity Component System Pattern

Lets say you've got a bicycle class.  It has a gear data member.  And you later decide it needs to have a location too, so you create a location base class and have the bicycle inherit from location.  Seems reasonable doesn't it?  Unless you aren't always concerned with location of the bicycle.

ECS allows attaching concerns to an object dynamically.  It might help to first look at how ECS is implemented and processed in time.

In terms of time organization, the ECS pattern processes concerns in a loop, just like the old main event loop processed events.  An entity can have multiple class instances attached to it, where each instance represents a concern, and those instances can, but don't have to be, attached at run-time.

For this to work, the entity has to process through a list of all attached instances and run an update method (and maybe others).  Common methods can be called directly using polymorphism, but message sending to custom methods may require reflection, or a more sophisticated approach.

Because this architecture organizes by concerns that entity is responsible for, the overall code organization tends to avoid class inheritance.  That seems like a really big deal, and if you've never had exposure to using ECS, might expose assumptions about how code needs to be organized.

Thus a concern (the location of the bicycle) is not inherited by a bicycle class, even if bicycles have location.  Instead a bicycle will have a location concern possibly attached to it.

There are always trade-offs, and workflow for architectural design might be affected.  If you don't have to worry about inheritance as much, writing code can be more straightforward.  Also it seems easier to learn new code by visually seeing which classes are attached to entities.  But the penalty is in the ECS processing at run-time.