I figure that I should edit in the following circumstances:
Before the commenting feature was added, I noticed that some moderators edit posts in order to insert a reply or extension to the post. I feel this isn't good etiquette (but I could be wrong) because I don't know what to do with their changes when I come to re-edit or respond.
I also feel that I should be very careful not to change the meaning of any question or answer, even if that meaning is misguided.
What other rules would be best practice?
We are implementing an emulator for a piece of hardware that
is being developed at the same time. The idea is to give 3rd parties
a software solution to test their client software and give the hardware
developers a reference point to implement their firmware.
The people who wrote the protocol for the hardware used a custom
version of SUN XDR called INCA_XDR. It's a tool to serialize and
de-serialize messages. It's written in C and we want to avoid any
native code so we are parsing the protocol data manually.
The protocol is by nature rather complex and the data packets
can have many different structures, but it always has the same global structure:
[HEAD] [INTRO] [DATA] [TAIL]
[HEAD] =
byte sync 0x03
byte length X [MSB] X = length of [HEADER] + [INTRO] + [DATA]
byte length X [LSB] X = length of [HEADER] + [INTRO] + [DATA]
byte check X [MSB] X = crc of [INTRO] [DATA]
byte check X [LSB] X = crc of [INTRO] [DATA]
byte headercheck X X = XOR over [SYNC] [LENGTH] [CHECK]
[INTRO]
byte version 0x03
byte address X X = 0 for point-to-point, 1-254 for specific controller, 255 = broadcast
byte sequence X X = sequence number
byte group X [MSB] X = The category of the message
byte group X [LSB] X = The category of the message
byte type X [MSB] X = The id of the message
byte type X [LSB] X = The id of the message
[DATA] =
The actuall data for the specified message,
this format really differs a lot.
It always starts with a DRCode which is one byte.
It more or less specifies the general structure of
the data, but even within the same structure the data
can mean many different things and have different lenghts.
(I think this is an artifact of the INCA_XDR tool)
[TAIL] =
byte 0x0D
As you can see there is a lot of overhead data, but this is because
the protocol needs to work with both RS232 (point-to-multipoint) and TCP/IP (p2p).
name size value
drcode 1 1
name 8 contains a name that can be used as a file name (only alphanumeric characters allowed)
timestamp 14 yyyymmddhhmmss contains timestamp of bitmap library
size 4 size of bitmap library to be loaded
options 1 currently no options
Or it might have an entirely different structure:
name size value
drcode 1 2
lastblock 1 0 - 1 1 indicates last block. Firmware can be stored
blocknumber 2 Indicates block of firmware
blocksize 2 N size of block to load
blockdata N data of block of firmware
Sometimes it's just a DRCode and no additional data.
Based on the group and the type field, the emulator
needs to perform certain actions. So first we look at those
two fields and based on that we know what to expect of the data
and have to parse it properly.
Then the response data needs to be generated which again has
many different data structures. Some messages simply generate
an ACK or NACK message, while others generate a real reply with data.
We decided to break things up in small pieces.
First of all there is the IDataProcessor.
Classes implementing this interface are responsible
for validating raw data and generating instances of the Message class.
They are not responsible for commmunication, they are simply passed a byte[]
Raw data validation means checking the header for checksum, crc and length errors.
The resulting message gets passed to a class that implements IMessageProcessor.
Even if the raw data was considered invalid, because the IDataProcessor has no
notion of response messages or anything else, all it does is validate the raw data.
To inform the IMessageProcessor about errors, some additional properties have been added
to the Message class:
bool nakError = false;
bool tailError = false;
bool crcError = false;
bool headerError = false;
bool lengthError = false;
They are not related to the protocol and only exist for the IMessageProcessor
The IMessageProcessor is where the real work is done.
Because of all the different message groups and types I decided to
use F# to implement the IMessageProcessor interface because pattern matching
seemed like a good way to avoid lots of nested if/else and caste statements.
(I have no prior experience with F# or even functional languages other than LINQ and SQL)
The IMessageProcessor analyzes the data and decides what methods it should call
on the IHardwareController. It might seem redundant to have IHardwareController,
but we want to be able to swap it out with a different implementation
and not be forced to use F# either. The current implementation is a WPF windows,
but it might be a Cocoa# window or simply a console for example.
The IHardwareController is also responsible for managing state because
the developers should be able to manipulate hardware parameters and errors through the user interface.
So once the IMessageProcessor has called the correct methods on IHardwareController,
it has to generate the response MEssage. Again... the data in these response messages
can have many different structures.
Eventually an IDataFactory is used to convert the Message to raw protocol data
ready to be sent to whatever class is responsible for communication.
(Additional encapsulation of the data might be required for example)
This is nothing "hard" about writing this code, but all the different
commands and data structures require lots and lots of code and there are few
things we can reuse. (At least as far as I can see now, hoping someone can prove me wrong)
This is the first time I use F#, so I'm actually learning as I go. The code below is far from finished
and probably looks like a giant mess. It only implements a handfull of all the messages in the protocol
and I can tell you there are lots and lots of them. So this file is going to get huge!
Important to know: the byte order is reversed over the wire (historical reasons)
module Arendee.Hardware.MessageProcessors
open System;
open System.Collections
open Arendee.Hardware.Extenders
open Arendee.Hardware.Interfaces
open System.ComponentModel.Composition
open System.Threading
open System.Text
let VPL_NOERROR = (uint16)0
let VPL_CHECKSUM = (uint16)1
let VPL_FRAMELENGTH = (uint16)2
let VPL_OUTOFSEQUENCE = (uint16)3
let VPL_GROUPNOTSUPPORTED = (uint16)4
let VPL_REQUESTNOTSUPPORTED = (uint16)5
let VPL_EXISTS = (uint16)6
let VPL_INVALID = (uint16)7
let VPL_TYPERROR = (uint16)8
let VPL_NOTLOADING = (uint16)9
let VPL_NOTFOUND = (uint16)10
let VPL_OUTOFMEM = (uint16)11
let VPL_INUSE = (uint16)12
let VPL_SIZE = (uint16)13
let VPL_BUSY = (uint16)14
let SYNC_BYTE = (byte)0xE3
let TAIL_BYTE = (byte)0x0D
let MESSAGE_GROUP_VERSION = 3uy
let MESSAGE_GROUP = 701us
[<Export(typeof<IMessageProcessor>)>]
type public StandardMessageProcessor() = class
let mutable controller : IHardwareController = null
interface IMessageProcessor with
member this.ProcessMessage m : Message =
printfn "%A" controller.Status
controller.Status <- ControllerStatusExtender.DisableBit(controller.Status,ControllerStatus.Nak)
match m with
| m when m.LengthError -> this.nakResponse(m,VPL_FRAMELENGTH)
| m when m.CrcError -> this.nakResponse(m,VPL_CHECKSUM)
| m when m.HeaderError -> this.nakResponse(m,VPL_CHECKSUM)
| m -> this.processValidMessage m
| _ -> null
member public x.HardwareController
with get () = controller
and set y = controller <- y
end
member private this.processValidMessage (m : Message) =
match m.Intro.MessageGroup with
| 701us -> this.processDefaultGroupMessage(m);
| _ -> this.nakResponse(m, VPL_GROUPNOTSUPPORTED);
member private this.processDefaultGroupMessage(m : Message) =
match m.Intro.MessageType with
| (1us) -> this.firmwareVersionListResponse(m) //ListFirmwareVersions 0
| (2us) -> this.StartLoadingFirmwareVersion(m) //StartLoadingFirmwareVersion 1
| (3us) -> this.LoadFirmwareVersionBlock(m) //LoadFirmwareVersionBlock 2
| (4us) -> this.nakResponse(m, VPL_FRAMELENGTH) //RemoveFirmwareVersion 3
| (5us) -> this.nakResponse(m, VPL_FRAMELENGTH) //ActivateFirmwareVersion 3
| (12us) -> this.nakResponse(m,VPL_FRAMELENGTH) //StartLoadingBitmapLibrary 2
| (13us) -> this.nakResponse(m,VPL_FRAMELENGTH) //LoadBitmapLibraryBlock 2
| (21us) -> this.nakResponse(m, VPL_FRAMELENGTH) //ListFonts 0
| (22us) -> this.nakResponse(m, VPL_FRAMELENGTH) //LoadFont 4
| (23us) -> this.nakResponse(m, VPL_FRAMELENGTH) //RemoveFont 3
| (24us) -> this.nakResponse(m, VPL_FRAMELENGTH) //SetDefaultFont 3
| (31us) -> this.nakResponse(m, VPL_FRAMELENGTH) //ListParameterSets 0
| (32us) -> this.nakResponse(m, VPL_FRAMELENGTH) //LoadParameterSets 4
| (33us) -> this.nakResponse(m, VPL_FRAMELENGTH) //RemoveParameterSet 3
| (34us) -> this.nakResponse(m, VPL_FRAMELENGTH) //ActivateParameterSet 3
| (35us) -> this.nakResponse(m, VPL_FRAMELENGTH) //GetParameterSet 3
| (41us) -> this.nakResponse(m, VPL_FRAMELENGTH) //StartSelfTest 0
| (42us) -> this.returnStatus(m) //GetStatus 0
| (43us) -> this.nakResponse(m, VPL_FRAMELENGTH) //GetStatusDetail 0
| (44us) -> this.ResetStatus(m) //ResetStatus 5
| (45us) -> this.nakResponse(m, VPL_FRAMELENGTH) //SetDateTime 6
| (46us) -> this.nakResponse(m, VPL_FRAMELENGTH) //GetDateTime 0
| _ -> this.nakResponse(m, VPL_REQUESTNOTSUPPORTED)
(* The various responses follow *)
//Generate a NAK response
member private this.nakResponse (message : Message , error) =
controller.Status <- controller.Status ||| ControllerStatus.Nak
let intro = new MessageIntro()
intro.MessageGroupVersion <- MESSAGE_GROUP_VERSION
intro.Address <- message.Intro.Address
intro.SequenceNumber <- this.setHigh(message.Intro.SequenceNumber)
intro.MessageGroup <- MESSAGE_GROUP
intro.MessageType <- 130us
let errorBytes = UShortExtender.ToIntelOrderedByteArray(error)
let data = Array.zero_create(5)
let x = this.getStatusBytes
let y = this.getStatusBytes
data.[0] <- 7uy
data.[1..2] <- this.getStatusBytes
data.[3..4] <- errorBytes
let header = this.buildHeader intro data
let message = new Message()
message.Header <- header
message.Intro <- intro
message.Tail <- TAIL_BYTE
message.Data <- data
message
//Generate an ACK response
member private this.ackResponse (message : Message) =
let intro = new MessageIntro()
intro.MessageGroupVersion <- MESSAGE_GROUP_VERSION
intro.Address <- message.Intro.Address
intro.SequenceNumber <- this.setHigh(message.Intro.SequenceNumber)
intro.MessageGroup <- MESSAGE_GROUP
intro.MessageType <- 129us
let data = Array.zero_create(3);
data.[0] <- 0x05uy
data.[1..2] <- this.getStatusBytes
let header = this.buildHeader intro data
message.Header <- header
message.Intro <- intro
message.Tail <- TAIL_BYTE
message.Data <- data
message
//Generate a ReturnFirmwareVersionList
member private this.firmwareVersionListResponse (message : Message) =
//Validation
if message.Data.[0] <> 0x00uy then
this.nakResponse(message,VPL_INVALID)
else
let intro = new MessageIntro()
intro.MessageGroupVersion <- MESSAGE_GROUP_VERSION
intro.Address <- message.Intro.Address
intro.SequenceNumber <- this.setHigh(message.Intro.SequenceNumber)
intro.MessageGroup <- MESSAGE_GROUP
intro.MessageType <- 132us
let firmwareVersions = controller.ReturnFirmwareVersionList();
let firmwareVersionBytes = BitConverter.GetBytes((uint16)firmwareVersions.Count) |> Array.rev
//Create the data
let data = Array.zero_create(3 + (int)firmwareVersions.Count * 27)
data.[0] <- 0x09uy //drcode
data.[1..2] <- firmwareVersionBytes //Number of firmware versions
let mutable index = 0
let loops = firmwareVersions.Count - 1
for i = 0 to loops do
let nameBytes = ASCIIEncoding.ASCII.GetBytes(firmwareVersions.[i].Name) |> Array.rev
let timestampBytes = this.getTimeStampBytes firmwareVersions.[i].Timestamp |> Array.rev
let sizeBytes = BitConverter.GetBytes(firmwareVersions.[i].Size) |> Array.rev
data.[index + 3 .. index + 10] <- nameBytes
data.[index + 11 .. index + 24] <- timestampBytes
data.[index + 25 .. index + 28] <- sizeBytes
data.[index + 29] <- firmwareVersions.[i].Status
index <- index + 27
let header = this.buildHeader intro data
message.Header <- header
message.Intro <- intro
message.Data <- data
message.Tail <- TAIL_BYTE
message
//Generate ReturnStatus
member private this.returnStatus (message : Message) =
//Validation
if message.Data.[0] <> 0x00uy then
this.nakResponse(message,VPL_INVALID)
else
let intro = new MessageIntro()
intro.MessageGroupVersion <- MESSAGE_GROUP_VERSION
intro.Address <- message.Intro.Address
intro.SequenceNumber <- this.setHigh(message.Intro.SequenceNumber)
intro.MessageGroup <- MESSAGE_GROUP
intro.MessageType <- 131us
let statusDetails = controller.ReturnStatus();
let sizeBytes = BitConverter.GetBytes((uint16)statusDetails.Length) |> Array.rev
let detailBytes = ASCIIEncoding.ASCII.GetBytes(statusDetails) |> Array.rev
let data = Array.zero_create(statusDetails.Length + 5)
data.[0] <- 0x08uy
data.[1..2] <- this.getStatusBytes
data.[3..4] <- sizeBytes //Details size
data.[5..5 + statusDetails.Length - 1] <- detailBytes
let header = this.buildHeader intro data
message.Header <- header
message.Intro <- intro
message.Data <- data
message.Tail <- TAIL_BYTE
message
//Reset some status bytes
member private this.ResetStatus (message : Message) =
if message.Data.[0] <> 0x05uy then
this.nakResponse(message, VPL_INVALID)
else
let flagBytes = message.Data.[1..2] |> Array.rev
let flags = Enum.ToObject(typeof<ControllerStatus>,BitConverter.ToInt16(flagBytes,0)) :?> ControllerStatus
let retVal = controller.ResetStatus flags
if retVal <> 0x00us then
this.nakResponse(message,retVal)
else
this.ackResponse(message)
//StartLoadingFirmwareVersion (Ack/Nak)
member private this.StartLoadingFirmwareVersion (message : Message) =
if (message.Data.[0] <> 0x01uy) then
this.nakResponse(message, VPL_INVALID)
else
//Analyze the data
let name = message.Data.[1..8] |> Array.rev |> ASCIIEncoding.ASCII.GetString
let text = message.Data.[9..22] |> Array.rev |> Seq.map(fun x -> ASCIIEncoding.ASCII.GetBytes(x.ToString()).[0]) |> Seq.to_array |> ASCIIEncoding.ASCII.GetString
let timestamp = DateTime.ParseExact(text,"yyyyMMddHHmmss",Thread.CurrentThread.CurrentCulture)
let size = BitConverter.ToUInt32(message.Data.[23..26] |> Array.rev,0)
let overwrite =
match message.Data.[27] with
| 0x00uy -> false
| _ -> true
//Create a FirmwareVersion instance
let firmware = new FirmwareVersion();
firmware.Name <- name
firmware.Timestamp <- timestamp
firmware.Size <- size
let retVal = controller.StartLoadingFirmwareVersion(firmware,overwrite)
if retVal <> 0x00us then
this.nakResponse(message, retVal) //The controller denied the request
else
this.ackResponse(message);
//LoadFirmwareVersionBlock (ACK/NAK)
member private this.LoadFirmwareVersionBlock (message : Message) =
if message.Data.[0] <> 0x02uy then
this.nakResponse(message, VPL_INVALID)
else
//Analyze the data
let lastBlock =
match message.Data.[1] with
| 0x00uy -> false
| _true -> true
let blockNumber = BitConverter.ToUInt16(message.Data.[2..3] |> Array.rev,0)
let blockSize = BitConverter.ToUInt16(message.Data.[4..5] |> Array.rev,0)
let blockData = message.Data.[6..6 + (int)blockSize - 1] |> Array.rev
let retVal = controller.LoadFirmwareVersionBlock(lastBlock, blockNumber, blockSize, blockData)
if retVal <> 0x00us then
this.nakResponse(message, retVal)
else
this.ackResponse(message)
(* Helper methods *)
//We need to convert the DateTime instance to a byte[] understood by the device "yyyymmddhhmmss"
member private this.getTimeStampBytes (date : DateTime) =
let stringNumberToByte s = Byte.Parse(s.ToString()) //Casting to (byte) would give different results
let yearString = date.Year.ToString("0000")
let monthString = date.Month.ToString("00")
let dayString = date.Day.ToString("00")
let hourString = date.Hour.ToString("00")
let minuteString = date.Minute.ToString("00")
let secondsString = date.Second.ToString("00")
let y1 = stringNumberToByte yearString.[0]
let y2 = stringNumberToByte yearString.[1]
let y3 = stringNumberToByte yearString.[2]
let y4 = stringNumberToByte yearString.[3]
let m1 = stringNumberToByte monthString.[0]
let m2 = stringNumberToByte monthString.[1]
let d1 = stringNumberToByte dayString.[0]
let d2 = stringNumberToByte dayString.[1]
let h1 = stringNumberToByte hourString.[0]
let h2 = stringNumberToByte hourString.[1]
let min1 = stringNumberToByte minuteString.[0]
let min2 = stringNumberToByte minuteString.[1]
let s1 = stringNumberToByte secondsString.[0]
let s2 = stringNumberToByte secondsString.[1]
[| y1 ; y2 ; y3 ; y4 ; m1 ; m2 ; d1 ; d2 ; h1 ; h2 ; min1 ; min2 ; s1; s2 |]
//Sets the high bit of a byte to 1
member private this.setHigh (b : byte) : byte =
let array = new BitArray([| b |])
array.[7] <- true
let mutable converted = [| 0 |]
array.CopyTo(converted, 0);
(byte)converted.[0]
//Build the header of a Message based on Intro + Data
member private this.buildHeader (intro : MessageIntro) (data : byte[]) =
let headerLength = 7;
let introLength = 7;
let length = (uint16)(headerLength + introLength + data.Length)
let crcData = ByteArrayExtender.Concat(intro.GetRawData(),data)
let crcValue = ByteArrayExtender.CalculateCRC16(crcData)
let lengthBytes = UShortExtender.ToIntelOrderedByteArray(length);
let crcValueBytes = UShortExtender.ToIntelOrderedByteArray(crcValue);
let headerChecksum = (byte)(SYNC_BYTE ^^^ lengthBytes.[0] ^^^ lengthBytes.[1] ^^^ crcValueBytes.[0] ^^^ crcValueBytes.[1])
let header = new MessageHeader();
header.Sync <- SYNC_BYTE
header.Length <- length
header.HeaderChecksum <- headerChecksum
header.DataChecksum <- crcValue
header
member private this.getStatusBytes =
let l = controller.Status
let status = (uint16)controller.Status
let statusBytes = BitConverter.GetBytes(status);
statusBytes |> Array.rev
end
(Please note that in the real source, the classes have different names, more specific than "Hardware")
I'm hoping for suggestions, ways to improve the code or even different ways to handle the problem.
For example, would the use of a dynamic language such as IronPython make things easier,
am I going at the the wrong way all together. What's your experience with problems like this,
what would you change, avoid, etc....
Update:
Based on the answer by Brian, I written down the following:
type DrCode9Item = {Name : string ; Timestamp : DateTime ; Size : uint32; Status : byte}
type DrCode11Item = {Id : byte ; X : uint16 ; Y : uint16 ; SizeX : uint16 ; SizeY : uint16
Font : string ; Alignment : byte ; Scroll : byte ; Flash : byte}
type DrCode12Item = {Id : byte ; X : uint16 ; Y : uint16 ; SizeX : uint16 ; SizeY : uint16}
type DrCode14Item = {X : byte ; Y : byte}
type DRType =
| DrCode0 of byte
| DrCode1 of byte * string * DateTime * uint32 * byte
| DrCode2 of byte * byte * uint16 * uint16 * array<byte>
| DrCode3 of byte * string
| DrCode4 of byte * string * DateTime * byte * uint16 * array<byte>
| DrCode5 of byte * uint16
| DrCode6 of byte * DateTime
| DrCode7 of byte * uint16 * uint16
| DrCode8 of byte * uint16 * uint16 * uint16 * array<byte>
| DrCode9 of byte * uint16 * array<DrCode9Item>
| DrCode10 of byte * string * DateTime * uint32 * byte * array<byte>
| DrCode11 of byte * array<DrCode11Item>
| DrCode12 of byte * array<DrCode12Item>
| DrCode13 of byte * uint16 * byte * uint16 * uint16 * string * byte * byte
| DrCode14 of byte * array<DrCode14Item>
I could continue doing this for all the DR types (quite a few),
but I still don't understand how that would help me. I've read
about it on Wikibooks and in Foundations of F# but something is not clicking in my head yet.
Update 2
So, I understand I could do the following:
let execute dr =
match dr with
| DrCode0(drCode) -> printfn "Do something"
| DrCode1(drCode, name, timestamp, size, options) -> printfn "Show the size %A" size
| _ -> ()
let date = DateTime.Now
let x = DrCode1(1uy,"blabla", date, 100ul, 0uy)
But when the message comes into the IMessageProcessor,
the choise is made right there what kind of message it is
and the proper function is then called. The above would just
be additional code, at least that is how understand it,
so I must really be missing the point here... but I don't see it.
execute x
]]>Contents:
[Vector, __add__, reflect_y, rotate, dilate, transform]
[Matrix, __add__, __str__, __mul__, zero, det, inv, __pow__]
Preface: Based on my teaching experience, I think that the courses referenced by others are very good courses. That means if your goal is understanding matrices as mathematicians do, than you should by all means get the whole course. But if your goals are more modest, here's my try at something more tailored to your needs (but still written with the goal to convey many theoretical concepts, kind of contradicting my original advice.)
How to use:
Before matrices come vectors. You sure know how to handle the 2- and 3- dimensional vectors:
class Vector:
"""This will be a simple 2-dimensional vector.
In case you never encountered Python before, this string is a
comment I can put on the definition of the class or any function.
It's just one of many cool features of Python, so learn it here!
"""
def __init__(self, x, y):
self.x = x
self.y = y
now you can write
v = Vector(5, 3)
w = Vector(7, -1)
but it's not much fun by itself. Let's add more useful methods:
def __str__(self: 'vector') -> 'readable form of vector':
return '({0}, {1})'.format(self.x, self.y)
def __add__(self:'vector', v: 'another vector') -> 'their sum':
return Vector(self.x + v.x, self.y + v.y)
def __mul__(self:'vector', number: 'a real number') -> 'vector':
'''Multiplies the vector by a number'''
return Vector(self.x * number, self.y * number)
That makes things more interesting as we can now write:
print(v + w * 2)
and get the answer (19, 1)
nicely printed as a vector (if the examples look unfamiliar, think how this code would look in C++).
Now it's all cool to be able to write 1274 * w
but you need more vector operations for the graphics. Here are some of them: you can flip the vector around (0,0)
point, you can reflect it around x
or y
axis, you can rotate it clockwise or counterclockwise (it's a good idea to draw a picture here).
Let's do some simple operations:
...
def flip(self:'vector') -> 'vector flipped around 0':
return Vector(-self.x, -self.y)
def reflect_x(self:'vector') -> 'vector reflected around x axis':
return Vector(self.x, -self.y)
print(v.flip(), v.reflect_x())
flip(...)
using the operations I had below? What about reflect_x
?Now you may wonder why I omitted reflect_y
. Well, it's because I want you to stop for a moment and write your own version of it. Ok, here's mine:
def reflect_y(self:'vector') -> 'vector reflected around y axis':
return self.flip().reflect_x()
See, if you look how this function computes, it's actually quite trivial. But suddenly an amazing thing happened: I was able to write a transformation using only the existing transformations flip
and reflect_x
. For all I care, reflect_y
could be defined in a derived class without access to x
and y
and it would still work!
Mathematicians would call these functions operators. They would say that reflect_y
is an operator obtained by composition of operators flip
and reflect_x
which is
denoted by reflect_y = flip ? reflect_x
(you should see the small circle, a Unicode symbol 25CB
).
=
symbol from now to denote that two operations produce the same result, like in the paragraph above. This is a "mathematical =
", which cannot be expressed as a program.So if I do
print(v.reflect_y())
I get the result (-5, 3)
. Go and picture it!
reflect_y ? reflect_y
. How would you name it?Those operations were nice and useful, but you are probably wondering why am so slow to introduce rotations. Ok, here I go:
def rotate(self:'vector', angle:'rotation angle') -> 'vector':
??????
At this point, if you know how to rotate vectors, you should go on and fill in the question marks. Otherwise please bear with me for one more simple case: counterclockwise rotation by 90
degrees. This one is not hard to draw on a piece of paper:
def rotate_90(self:'vector') -> 'rotated vector':
new_x = - self.y
new_y = self.x
return Vector(new_x, new_y)
Trying
x_axis = Vector(1, 0)
y_axis = Vector(0, 1)
print(x_axis.rotate_90(), y_axis.rotate_90())
now gives (0, 1) (-1, 0)
. Run it yourself!
flip = rotate_90 ? rotate_90
.Anyway, I won't hide the secret ingredient for longer:
import math # we'll need math from now on
...
class Vector:
...
def rotate(self:'vector', angle:'rotation angle') -> 'rotated vector':
cos = math.cos(angle)
sin = math.sin(angle)
new_x = cos * self.x - sin * self.y
new_y = sin * self.x + cos * self.y
return Vector(new_x, new_y)
Now let's try something along the lines:
print(x_axis.rotate(90), y_axis.rotate(90))
If you expect the same result as before, (0, 1) (-1, 0)
, you're bound to be disappointed. That code prints:
(-0.448073616129, 0.893996663601) (-0.893996663601, -0.448073616129)
and boy, is it ugly!
Notation: I will say that we applied operation rotate(90)
to x
in the example above. The knowledge we gained is that rotate(90) != rotate_90
.
Question: What happened here? How to express rotate_90
in terms of rotate
? How to express flip
in terms of rotate
?
Those rotations are certainly useful, but they are not everything you need to do even the 2D graphics. Consider the following transformations:
def dilate(self:'vector', axe_x:'x dilation', axe_y:'y dilation'):
'''Dilates a vector along the x and y axes'''
new_x = axe_x * self.x
new_y = axe_y * self.y
return Vector(new_x, new_y)
This dilate
thing dilates the x
and y
axes in a possibly different way.
dilate(?, ?) = flip
, dilate(?, ?) = reflect_x
.I will use this dilate
function to demonstrate a thing mathematicians call commutativity: that is, for every value of parameters a
, b
, c
, d
you can be sure that
dilate(a, b) ? dilate(c, d) = dilate(c, d) ? dilate(a, b)
Exercise: Prove it. Also, is it true that for all possible values of parameters those below would hold?
`rotate(a) ? rotate(b) = rotate(b) ? rotate(a)`
`dilate(a, b) ? rotate(c) = rotate(c) ? dilate(a, b)`
`rotate(a) ? __mul__(b) = __mul__(b) ? rotate(a)`
Let's summarize all the stuff we had around here, our operators on vector x
flip
, reflect_x
, *
, rotate(angle)
, dilate(x, y)
from which one could make some really crazy stuff like
flip ? rotate(angle) ? dilate(x, y) ? rotate(angle_2) ? reflect_y + reflect_x = ???
As you create more and more complicated expressions, one would hope for some kind of order that would suddenly reduce all possible expressions to a useful form. Fear not! Magically, every expression of the form above can be simplified to
def ???(self:'vector', parameters):
'''A magical representation of a crazy function'''
new_x = ? * self.x + ? * self.y
new_y = ? * self.x + ? * self.y
return Vector(new_x, new_y)
with some numbers and/or parameters instead of ?
s.
__mul__(2) ? rotate(pi/4)
dilate(x, y) ? rotate(pi/4)
This allows us to write a universal function
def transform(self:'vector', m:'matrix') -> 'new vector':
new_x = m[0] * self.x + m[1] * self.y
new_y = m[2] * self.x + m[3] * self.y
return Vector(new_x, new_y)
which would take any 4-tuple of numbers, called matrix, and apply it to vector x
. Here's an example:
rotation_90_matrix = (0, -1, 1, 0)
print(v, v.rotate_90(), v.transform(rotation_90_matrix))
which prints (5, 3) (-3, 5) (-3, 5)
. Note that if you apply transform
with
any matrix to origin, you still get origin:
origin = Vector(0, 0)
print(origin.transform(rotation_90_matrix))
m
that describe flip
, dilate(x, y)
, rotate(angle)
?As we part with the Vector
class, here's an exercise for those who want to test both their vector math knowledge and Pythonic skills:
Vector
class all vector operations that you can come up with (how many of standard operators can you overload for vectors? Check out my answer).As we found out in the previous section, a matrix can be thought of a shorthand that allows us to encode a vector operation in a simple way. For example, rotation_90_matrix
encodes the rotation by 90 degrees.
Now as we shift our attention from vectors to matrices, we should by all means have a class
for matrix as well. Moreover, in that function Vector.transform(...)
above the role of the matrix was somewhat misrepresented. It's more usual for m
to be fixed while vector changes, so from now on our transformations will be methods of matrix class:
class Matrix:
def __init__(self:'new matrix', m:'matrix data'):
'''Create a new matrix.
So far a matrix for us is just a 4-tuple, but the action
will get hotter once The (R)evolution happens!
'''
self.m = m
def __call__(self:'matrix', v:'vector'):
new_x = self.m[0] * v.x + self.m[1] * v.y
new_y = self.m[2] * v.x + self.m[3] * v.y
return Vector(new_x, new_y)
If you don't know Python, __call__
overloads the meaning of (...)
for matrices so I can use the standard notation for a matrix acting on a vector. Also, the matrices are usually written using a single uppercase letter:
J = Matrix(rotation_90_matrix)
print(w, 'rotated is', J(w))
Now, let's find out what else we can do with matrices. Remember that matrix m
is really just a way to encode an operaton on vectors. Note that for two functions m1(x)
and m2(x)
I can create a new function (using lambda notation) m = lambda x: m1(x) + m2(x)
. It turns out if m1
and m2
were enconded by matrices, you can also encode this m
using matrices!
You just have to add its data, like (0, 1, -1, 0) + (0, 1, -1, 0) = (0, 2, -2, 0)
. Here's how to add two tuples in Python, with some very useful and highly Pythonic techniques:
def __add__(self:'matrix', snd:'another matrix'):
"""This will add two matrix arguments.
snd is a standard notation for second argument.
(i for i in array) is Python's powerful list comprehension.
zip(a, b) is used to iterate over two sequences together
"""
new_m = tuple(i + j for i, j in zip(self.m, snd.m))
return Matrix(new_m)
Now we can write expressions like J + J
or even J + J + J
, but to see the results we have to figure out how to print a Matrix. A possible way would be to print a 4-tuple of numbers, but let's take a hint from the Matrix.__call__
function that the numbers should be organized into a 2x2
block:
def as_block(self:'matrix') -> '2-line string':
"""Prints the matrix as a 2x2 block.
This function is a simple one without any advanced formatting.
Writing a better one is an exercise.
"""
return ('| {0} {1} |\n' .format(self.m[0], self.m[1]) +
'| {0} {1} |\n' .format(self.m[2], self.m[3]) )
If you look at this function in action you'll notice there is some room for improvement:
print((J + J + J).as_block())
Matrix.__str__
that will round theNow you should be able to write the matrix for rotation:
def R(a: 'angle') -> 'matrix of rotation by a':
cos = math.cos(a)
sin = math.sin(a)
m = ( ????? )
return Matrix(m)
Exercise: Examine the code for Vector.rotate(self, angle)
and fill in the question marks. Test with
from math import pi
print(R(pi/4) + R(-pi/4))
The most important thing we can do with one-parameter functions is compose them: f = lambda v: f1(f2(v))
. How to mirror that with matrices? This requires us to examine how Matrix(m1) ( Matrix(m2) (v))
works. If you expand it, you'll notice that
m(v).x = m1[0] * (m2[0]*v.x + m2[1]*v.y) + m1[1] * (m2[2]*v.x + m2[3]*v.y)
and similarly for m(v).y
, which, if you open the parentheses, looks suspiciously similar
to Matrix.__call__
using a new tuple m
, such that m[0] = m1[0] * m2[0] + m1[2] * m2[2]
. So let's take this as a hint for a new definiton:
def compose(self:'matrix', snd:'another matrix'):
"""Returns a matrix that corresponds to composition of operators"""
new_m = (self.m[0] * snd.m[0] + self.m[1] * snd.m[2],
self.m[0] * snd.m[1] + self.m[1] * snd.m[3],
???,
???)
return Matrix(new_m)
Exercise: Fill in the question marks here. Test it with
print(R(1).compose(R(2)))
print(R(3))
Math exercise: Prove that R(a).compose(R(b))
is always the same as R(a + b)
.
Now let me tell the truth: this compose
function is actually how mathematicians decided to multiply matrices.
This makes sense as a notation: A * B
is a matrix that decribes operator A ? B
, and as we'll see next there are deeper reasons to call this 'multiplication' as well.
To start using multiplication in Python all we have to do is to order it so in a Matrix
class:
class Matrix:
...
__mul__ = compose
(R(pi/2) + R(pi)) * (R(-pi/2) + R(pi))
. Try to find the answer yourself first on a piece of paper.+
and *
Let's make some good name for the matrix that corresponds to the dilate(a, b)
operator. Now there's nothing wrong with D(a, b)
, but I'll
use a chance to introduce a standard notation:
def diag(a: 'number', b: 'number') -> 'diagonal 2x2 matrix':
m = (a, 0, 0, b)
return Matrix(m)
Try print(diag(2, 12345))
to see why it's called a diagonal matrix.
As the composition of operations was found before to be not always commutative, *
operator won't be always commutative for matrices either.
A
, B
, made from R
and diag
,A * B
is not equal to B * A
.This is somewhat strange, since multiplication for numbers is always commutative, and raises the question whether compose
really deserves to be called __mul__
. Here's quite a lot of rules that +
and *
do satisfy:
A + B = B + A
A * (B + C) = A * B + A * C
(A + B) * C = A * C + B * C
(A * B) * C = A * (B * C)
A - B
and (A - B) + B = A
A - B
in terms of +
, *
and diag
? What does A - A
equal to? Add the method __sub__
to the class Matrix
. What happens if you compute R(2) - R(1)*R(1)
? What should it be equal to?The (A * B) * C = A * (B * C)
equality is called associativity and is especially nice since it means that we don't have to worry about putting parentheses in an expression
of the form A * B * C
:
print(R(1) * (diag(2,3) * R(2)))
print((R(1) * diag(2,3)) * R(2))
Let's find analogues to regular numbers 0
and 1
and subtraction:
zero = diag(0, 0)
one = diag(1, 1)
With the following easily verifiable additions:
A + zero = A
A * zero = zero
A * one = one * A = A
the rules become complete, in the sense that there is a short name for them: ring axioms.
Mathematicians thus would say that matrices form a ring, and they indeed always use symbols +
and *
when talking about rings, and so shall we.
Using the rules it's possible to easily compute the expression from the previous section:
(R(pi/2) + R(pi)) * (R(-pi/2) + R(pi)) = R(pi/2) * R(-pi/2) + ... = one + ...
(R(a) + R(b)) * (R(a) - R(b)) = R(2a) - R(2b)
.Time to return to how we defined matrices: they are a shortcut to some operations you can do with vectors, so it's something you can actually draw. You might want to take a pen or look at the materials that others suggested to see examples of different plane transformations.
Among the transformations we'll be looking for the affine ones, those who look 'the same' everywhere (no bending). For example, a rotation around some point (x, y)
qualifies. Now this one cannot be expressed as lambda v: A(v)
, but in can be written in the form lambda v: A(v) + b
for some matrix A
and vector b
.
A
and b
such that a rotation by pi/2
around the point (1, 0)
has the form above. Are they unique?Note that for every vector there is an affine transformation which is a shift by the vector.
An affine transformation may stretch or dilate shapes, but it should do in the same way everywhere. Now I hope you believe that the area of any figure changes by a constant number under the transformation. For a transformation given by matrix A
this coeffiecient is called the determinant of A
and can be computed applying the formula for an area to two vectors A(x_axis)
and A(y_axis)
:
def det(self: 'matrix') -> 'determinant of a matrix':
return self.m[0]*self.m[3] - self.m[1] * self.m[2]
As a sanity check, diag(a, b).det()
is equal to a * b
.
As you can see, the determinant of rotation matrix is always the same:
from random import random
r = R(random())
print (r, 'det =', r.det())
One interesting thing about det
is that it is multiplicative (it kind of follows from the definition if you meditate long enough):
A = Matrix((1, 2, -3, 0))
B = Matrix((4, 1, 1, 2))
print(A.det(), '*', B.det(), 'should be', (A * B).det())
A useful thing you can do with matrices is write a system of two linear equations
A.m[0]*v.x + A.m[1]*v.y = b.x
A.m[2]*v.x + A.m[3]*v.y = b.y
in a simpler way: A(v) = b
. Let's solve the system as they teach in (some) high schools: multiply first equation by A.m[3]
, second by -A.m1 and add (if in doubt, do this on a piece of paper) to solve for v.x
.
If you really tried it, you should have got A.det() * v.x = (A.m[3]) * b.x + (-A.m[1]) * b.y
, which suggests that you can always get v
by multiplying b
by some other matrix. This matrix is called inverse of A
:
def inv(self: 'matrix') -> 'inverse matrix':
'''This function returns an inverse matrix when it exists,
or raises ZeroDivisionError when it doesn't.
'''
new_m = ( self.m[3] / self.det(), -self.m[1] / self.det(),
????? )
return Matrix(new_m)
As you see, this method fails loudly when determinant of matrix is zero. If you really want you can catch this expection with:
try:
print(zero.inv())
except ZeroDivisionError as e: ...
self.det() == 0
. Write the method to divide matrices and test it. Use the inverse matrix to solve an equation A(v) = x_axis
(A
was defined above).The main property of inverse matrix is that A * A.inv()
always equals to one
That's why mathematicians denote A.inv()
by A
<sup>-1</sup>. How about we write a
nice function to use A ** n
notation for A
<sup>n</sup>? Note that the naive for i in range(n): answer *= self
cycle is O(|n|) which is certainly too slow, because
this can be done with a complexity of log |n|
:
def __pow__(self: 'matrix', n:'integer') -> 'n-th power':
'''This function returns n-th power of the matrix.
It does it more efficiently than a simple for cycle. A
while loop goes over all bits of n, multiplying answer
by self ** (2 ** k) whenever it encounters a set bit.
...
Exercise: Fill in the details in this function. Test it with
X, Y = A ** 5, A ** -5
print (X, Y, X * Y, sep = '\n')
This function only works for integer values of n
, even though for some matrices we can also define a fractional power, such as square root (in other words, a matrix B
such that B * B = A
).
diag(-1, -1)
. Is this the only possible answer?Here I'm going to introduce you to the subject in exactly one section!
Since it's a complex subject, I'm likely to fail, so please forgive me in advance.
First, similarly to how we have matrices zero
and one
, we can make a matrix out of any real number by doing diag(number, number)
. Matrices of that form can be added, subtracted, multiplied, inverted and the results would mimic what happens with the numbers themselves. So for all practical purposes, one can say that, e.g., diag(5, 5)
is 5.
However, Python doesn't know yet how to handle expressions of the form A + 1
or 5 * B
where A
and B
are matrices. If you're interested, you should by all means go and do the following exercise or look at my implementation (which uses a cool Python feature called decorator); otherwise, just know that it's been implemented.
Matrix
class so that in all standard operations where one of operands is a matrix and another a number, the number is automatically converted to the diag
matrix. Also add comparison for equality.Here's an example test:
print( 3 * A - B / 2 + 5 )
Now here's the first interesting complex number: the matrix J
, introduced in the beginning and equal to Matrix((0, 1, -1, 0))
, has a funny property that J * J == -1
(try it!). That means J
is certainly not a normal number, but, as I just said, matrices and numbers easily mix together. For example,
(1 + J) * (2 + J) == 2 + 2 * J + 1 * J + J * J = 1 + 3 * J
using the rules listed some time before. What happens if we test this in Python?
(1 + J) * (2 + J) == 1 + 3*J
That should happily say True
. Another example:
(3 + 4*J) / (1 - 2*J) == -1 + 2*J
As you might have guessed, the mathematicians don't call those 'crazy numbers', but they do something similar - they call expressions of the form a + b*J
complex numbers.
Because those are still instances of our Matrix
class, we can do quite a lot of operations with those: addition, subtraction, multiplication, division, power - it's all already implemented! Aren't matrices amazing?
I have overlooked the question of how to print the result of operation like E = (1 + 2*J) * (1 + 3*J)
so that it looks like an expression with J
rather than a 2x2
matrix. If you examine it carefully,
you'll see that you need to print the left column of that matrix in the format ... + ...J
(just one more nice thing: it's exactly E(x_axis)
!) Those who know the difference between str()
and repr()
should see it's natural to name a function that would produce expression of such form as repr()
.
Exercise: Write the function Matrix.__repr__
that would do exactly that and try some tests with it, like (1 + J) ** 3
, first computing the result on paper and then trying it with Python.
Math question: What is the determinant of a + b*J
? If you know what the absolute value of complex number is: how they are connected? What is the absolute value of a
? of a*J
?
In the final part of this trilogy we will see that everything is a matrix. We'll start with general M x N
matrices, and find out how vectors can be thought of as 1 x N
matrices and why numbers are the same as diagonal matrices. As a side note we'll explore the complex numbers as 2 x 2
matrices.
Finally, we will learn to write affine and projective transformations using matrices.
So the classes planned are [MNMatrix, NVector, Affine, Projective]
.
I guess if you was able to bear with me until here, you could be interested in this sequel, so I'd like to hear if I should continue with this (and where, since I'm pretty much sure I'm beyond what considered reasonable length of a single document).
]]>Advertisement
You will like those projects!
Enable typographer option to see result.
(c) (C) (r) (R) (tm) (TM) (p) (P) +-
test.. test... test..... test?..... test!....
!!!!!! ???? ,, -- ---
"Smartypants, double quotes" and 'single quotes'
This is bold text
This is bold text
This is italic text
This is italic text
Strikethrough
Blockquotes can also be nested...
...by using additional greater-than signs right next to each other...
...or with spaces between arrows.
Unordered
+
, -
, or *
Ordered
Lorem ipsum dolor sit amet
Consectetur adipiscing elit
Integer molestie lorem at massa
You can use sequential numbers...
...or keep all the numbers as 1.
Start numbering with offset:
Inline code
Indented code
// Some comments
line 1 of code
line 2 of code
line 3 of code
Block code "fences"
Sample text here...
Syntax highlighting
var foo = function (bar) {
return bar++;
};
console.log(foo(5));
Option | Description |
---|---|
data | path to data files to supply the data that will be passed into templates. |
engine | engine to be used for processing templates. Handlebars is the default. |
ext | extension to be used for dest files. |
Right aligned columns
Option | Description |
---|---|
data | path to data files to supply the data that will be passed into templates. |
engine | engine to be used for processing templates. Handlebars is the default. |
ext | extension to be used for dest files. |
Autoconverted link https://github.com/nodeca/pica (enable linkify to see)
Like links, Images also have a footnote style syntax
With a reference later in the document defining the URL location:
The killer feature of markdown-it
is very effective support of
syntax plugins.
Classic markup:
Shortcuts (emoticons): 8-)
see how to change output with twemoji.
++Inserted text++
==Marked text==
Footnote 1 link[^first].
Footnote 2 link[^second].
Inline footnote^[Text of inline footnote] definition.
Duplicated footnote reference[^second].
[^first]: Footnote can have markup
and multiple paragraphs.
[^second]: Footnote text.
Term 1
: Definition 1
with lazy continuation.
Term 2 with inline markup
: Definition 2
{ some code, part of Definition 2 }
Third paragraph of definition 2.
Compact style:
Term 1
~ Definition 1
Term 2
~ Definition 2a
~ Definition 2b
This is HTML abbreviation example.
It converts "HTML", but keep intact partial entries like "xxxHTMLyyy" and so on.
*[HTML]: Hyper Text Markup Language
::: warning
here be dragons
:::
This is what a topic and post looks like. As an administrator, you can edit the post's title and content.
To customise your forum, go to the Administrator Control Panel. You can modify all aspects of your forum there, including installation of third-party plugins.