Suggestion Box: Implementing Custom Gestures

"Today I try to implement custom Gesture on my application. I try to create the form that the user can customize Gesture for "Close" action at runtime, I don't know how to do this. Two week before I designed "C" gesture for "Close" action but the user want to design it himself.
Thank you.
-paracet-"

Great question. Go watch Hands-On: Gestures in the VCL by Seppy Bloom. That should answer all the questions you have about creating and using custom gestures in your applications.

Hacking the TTouchKeyboard Part IV

Hacking the TTouchKeyboard Part I
Hacking the TTouchKeyboard Part II
Hacking the TTouchKeyboard Part III
Hacking the TTouchKeyboard Part IV
Hacking the TTouchKeyboard Part V (not published yet)

Here is a quick off the top of my head documentation for the XML layout that is used by the keyboard compiler. If something doesn't behave as expected drop me a line with a comment or email so I can get it fixed.


This describes the schema and the attributes for the keyboard layout XML file.

<keyboard keyboardtype="Standard" keyboardname="Standard101" width=768" height="228" maxwidth="0" maxheight="0" minwidth="0" minheight="0">

keyboardtype - this allows multiple keyboard layouts to contribute to an overall keyboard type. For instance, for Standard we have the Standard101, Standard102 and Standard 106 keyboard layouts. Each one specifies a different layout for the Standard keyboard layout depending on the keyboard's language. This becomes the value of the TTouchKeyboard.KeyboardLayout property.

keyboardname - this is only used for humans to distinguish which keyboard layout is being used.

width - this is the default width of the keyboard layout.

height - is the default height of the keyboard layout.

maxwidth - is the maxwidth constraint of the keyboard layout.

minwidth - is the minwidth constraint of the keyboard layout.

maxheight - is the maxheight constraint of the keyboard layout.

minheight - is the minheight constraint of the keyboard layout.


<language languagename="en" />

languagename - specifies the two letter language code for the languages that this keyboard layout applies. When the system is searching for a keyboard layout for the current keyboard language it starts from the last keyboard layout registered, to allow developers to override the defaults, once a keyboardtype and language are found that keyboard layout is used.


<row topmargin="0" bottommargin="2">

topmargin - this is the padding between the key rows.

bottommargin - this is the padding between the key rows.


<key scancode="1" vk="200" caption="Esc" width="65" stretch="True" modifiername="fn" imageindex="3" toggle="True" mapscancode="42" mapvk="0" margin="5" doubletall="True" groupindex="1" combokeys="vk:12;vk:12;scancode:24;scancode:24">

scancode - this is the Scan Code that is sent to the active window. The default value is 0. A valid scan code is greater than zero. Any value greater than zero will be sent as a key. If the value is 0 then it is key that does not have a scan code. Any key that does not have a scan code and or a vk is not a real key. If the value is less than 0 then it is also a fake key. This is useful for modifier keys.

vk - Same as scancode above only for the vk. It is important to note that either one or both can be provided. If one isn't provided then the other will be determined by the OS.

caption - this is the key caption to display.

imageindex - this is a image that is displayed. -1 is the default which means the key caption is text. tab=0, shift=1, enter=2, backspace=3, up=4, down=5, left=6, right=7.

width - this is how wide the key is. Since the keyboard is flow layout from the upper left hand following to the right and down it just gets the next available location. The width and height of the key will be the hit test area. The button will be drawn inside this region offset by the margins.

height - Set the height of the key. The only valid value here is double the height of the row. Only one tall key is valid per row and only on the right hand side.

leftmargin - is the margin to the left of the key.

rightmargin - is the margin to the right of the key.

stretch - in flow layout when resizing setting the stretch to True will tell the resize function to allow this key to take up all available space. Keys that are not stretch keys must have their appropriate proportions no matter the size of the keyboard while keys that are stretch keys will take up the available space.

toggle - if a key is a toggle key then it can be pressed and stay's down. A second press will release the key and draw it normally.

groupindex - toggle keys can be in a group. If two ore more keys are in the same group only one of them can be toggled at a time. When pressing a toggle key all the other toggle keys in the same group will be drawn normally while the newly pressed key will be pressed.

combokeys - there are times when it is handy to generate a key sequence. The format is vk:# or scancode:# separated by a semicolon. The first time the vk or scancode is encountered it will trigger a down, the second occurrence it will be an up.

mapscancode, mapvk - keys like the left and right shift keys or left and right ctrl keys have different scan codes and vks, so to show the same state on these keys one can provide an alternate vk or scancode values.

modifiername - this tells the keyboard that this is a modifier key. When this key is toggled and a key has a modifier attribute then the key can change it's caption, scancode, vk and imageindex.

publishedname - this is the name of the key caption override that is displayed in the object inspector under the CaptionOverrides property. This caption will override any and all captions for this key. Note that the developer can provide keyboard locale specific overrides programatically.


<language languagename="fr" scancode="100" vk="2" imageindex="1" caption="AltGr" />

languagename - specifies a language override for the key. If the keyboard language is fr then the scancode, vk, imageindex and caption will overridden with the values provided.


<modifier scancode="44" vk="0" scancode="20" vk="20" imageindex="2" caption="foo" modifiername="caps">

modifiername - specifies an override for the key. If the modifier is toggled then the scancode, vk, imageindex and caption will overridden with the values provided.


<language languagename="fr" scancode="100" vk="2" imageindex="1" caption="AltGr" />
</modifier>
</key>
</row> </keyboard>


Hacking the TTouchKeyboard Part III

Hacking the TTouchKeyboard Part I
Hacking the TTouchKeyboard Part II
Hacking the TTouchKeyboard Part III
Hacking the TTouchKeyboard Part IV
Hacking the TTouchKeyboard Part V (not published yet)

This post is going to be short because it is just an example application to illustrate how to convert an XML touch keyboard layout the binary version that you can then add as a resource and use from your touch applications. You can download the source to the command line utility called kcc (stands for Keyboard Command line Compiler).

My next post will be the documentation for the XML layout and the final post on hacking the TTouchKeyboard will walk you through the steps to make this all work.

Hacking the TTouchKeyboard Part II

Hacking the TTouchKeyboard Part I
Hacking the TTouchKeyboard Part II
Hacking the TTouchKeyboard Part III
Hacking the TTouchKeyboard Part IV
Hacking the TTouchKeyboard Part V (not published yet)

This post is going to be short because it is just an example application that you can download source code to illustrate how to access and save a touch keyboard layout as an XML file.

My next post will be an example application illustrating generate a resource file from the XML file so you can make your own keyboard layouts.

Hacking the TTouchKeyboard Part I

Hacking the TTouchKeyboard Part I
Hacking the TTouchKeyboard Part II
Hacking the TTouchKeyboard Part III
Hacking the TTouchKeyboard Part IV
Hacking the TTouchKeyboard Part V (not published yet)

The TTouchKeyboard is a a fairly flexible on screen keyboard that supports multiple keyboard layouts but only shipped with the number pad and the standard keyboard layout for a whole lot of input languages. How to create these keyboard layouts has been a mystery, until now. Over the next few posts I'll provide some tools, documentation and sample code to build everything you need to hack the TTouchKeyboard.

The first bit of code I'm going to provide demonstrates listing all the keyboard layouts that are available. It's pretty simple really. A keyboard layout is just a resource with the string "KEYBOARD" in the name.

procedure GetKeyboardLayoutNames(ANames: TStrings);

function EnumResNames(Module: HMODULE; ResType, ResName: PChar;
Nothing: Pointer): Integer; stdcall;
const
sResourceName = 'KEYBOARD';
begin
if Pos(sResourceName, ResName) > 0 then
LayoutNames.Add(ResName);
Result := 1;
end;

begin
LayoutNames := TStringList.Create;
try
EnumResourceNames(HInstance, RT_RCDATA, @EnumResNames, 0);
ANames.Clear;
ANames.AddStrings(LayoutNames);
finally
FreeAndNil(LayoutNames);
end;
end;

Once you have a layout you can load it by calling this little function.

function LoadLayout(const LayoutName: string): TVirtualKeyLayout;
var
Stream: TResourceStream;
TempStream: TStream;
begin
Result := nil;
Stream := TResourceStream.Create(HInstance, LayoutName, RT_RCDATA);
try
Result := TVirtualKeyLayout.Create;
Result.LoadFromStream(Stream);
finally
Stream.Free;
end;
end;

Next I'll show how to iterate the data structures and save it to an XML file for easy text manipulation.

Touch Demo

My Touch Move demo is now complete. Here is a quick list of the four parts:

Part I
Part II
Part III
Part IV

And you can download source code from CodeCentral for all four demos. The download wasn't working but has now been fixed.

After the Thanksgiving Holiday I'll start something new. I was thinking of how to hack the touch keyboard, but if anyone has suggestions regarding things you'd like to know about touch drop me a line either via comments or the contact me link on the right side of this blog and I'll try and make a post out of it.

Touch Demo Part IV

Previously I posted Part I, Part II and Part III of the Touch Demo.

This is the final step of the touch demo where I will be adding the inertia processing and mouse support. Mouse support is easy but requires a minor refactoring of the touch message handler so the mouse can piggy back on it. For CodeRage I did add a touch keyboard but I will leave that as an exercise for the user.

And now you can download source code from CodeCentral for all four demos. Update: The download wasn't working but it has now been fixed.

Inertia processing requires the Manipulations unit. Then the object that is to be manipulated, in this case TGlowSpot, should implement the _IManipulationsEvents interface which is a COM event sync. Here is how that class changed:

  TGlowSpot = class(TInterfacedObject, _IManipulationEvents)
  private
    FInertia: IInertiaProcessor;
    FManipulator: IManipulationProcessor;
    FInertiaCookie, FManipulatorCookie: LongInt;
    FCompleted: BOOL;
  public
    X, Y, Radius: Integer;
    Alpha: Extended;
    FadeIn: Boolean;
    Color: TColor;
    ID: Integer;

  protected
    procedure DoTouch(const APoint: TPoint; AID: Integer; ATouchMessage: TTouchMessage);

    { _IManipulationEvents }
    function ManipulationStarted(X: Single; Y: Single): HRESULT; stdcall;
    function ManipulationDelta(X: Single; Y: Single; translationDeltaX: Single;
      translationDeltaY: Single; scaleDelta: Single; expansionDelta: Single;
      rotationDelta: Single; cumulativeTranslationX: Single;
      cumulativeTranslationY: Single; cumulativeScale: Single;
      cumulativeExpansion: Single; cumulativeRotation: Single): HRESULT;
      stdcall;
    function ManipulationCompleted(X: Single; Y: Single;
      cumulativeTranslationX: Single; cumulativeTranslationY: Single;
      cumulativeScale: Single; cumulativeExpansion: Single;
      cumulativeRotation: Single): HRESULT; stdcall;
  public
    constructor Create(AParent: TWinControl);

    procedure Paint(Canvas: TDirect2DCanvas);
    procedure Disconnect;
    procedure ProcessInertia;
  end;
...

Each TGlowSpot needs to hold on to an instance of the inertia processor and the manipulation processor and then connect up to them with the event sync. That is done in the constructor:

constructor TGlowSpot.Create(AParent: TWinControl);
begin
  inherited Create;
  Alpha := 1;
  Radius := 80;
  FadeIn := False;
  Randomize;
  Color := RGB(Random(255), Random(256), Random(256));
  ID := -1;
  FCompleted := True;
  ID := -1;
  FCompleted := True;

  FInertia := CreateComObject(CLSID_IInertiaProcessor) as IInertiaProcessor;
  FManipulator := CreateComObject(CLSID_IManipulationProcessor) as IManipulationProcessor;
  InterfaceConnect(FInertia, _IManipulationEvents, Self, FInertiaCookie);
  InterfaceConnect(FManipulator, _IManipulationEvents, Self, FManipulatorCookie);

  FInertia.put_DesiredDeceleration(0.001);

  FInertia.put_BoundaryLeft(200);
  FInertia.put_BoundaryTop(200);
  FInertia.put_BoundaryRight(AParent.Width - 200);
  FInertia.put_BoundaryBottom(AParent.Height - 200);

  FInertia.put_ElasticMarginLeft(200);
  FInertia.put_ElasticMarginTop(200);
  FInertia.put_ElasticMarginRight(200);
  FInertia.put_ElasticMarginBottom(200);
end;

And then of the disconnect is done like this:

procedure TGlowSpot.Disconnect;
begin
  InterfaceDisconnect(FInertia, _IManipulationEvents, FInertiaCookie);
  InterfaceDisconnect(FManipulator, _IManipulationEvents, FManipulatorCookie);
end;

The inertia processing is handled with each WM_TOUCH message that is fired. It's pretty simple really, if there is a move, call the manipulation COM object's ProcessMove, and do the same with down. Up is a bit different because of the inertia but all you need to do is set the velocity and the location.

procedure TGlowSpot.DoTouch(const APoint: TPoint; AID: Integer;
  ATouchMessage: TTouchMessage);
var
  Vx, Vy: Single;
begin
  case ATouchMessage of
    tmMove:
    begin
      X := APoint.X;
      Y := APoint.Y;
      FManipulator.ProcessMove(AID, APoint.X, APoint.Y);
    end;

    tmDown:
    begin
      X := APoint.X;
      Y := APoint.Y;
      FManipulator.ProcessDown(AID, APoint.X, APoint.Y);
    end;

    tmUp:
    begin
      ID := -1;

      FManipulator.ProcessUp(AID, APoint.X, APoint.Y);

      FManipulator.GetVelocityX(Vx);
      FManipulator.GetVelocityY(Vy);

      FInertia.put_InitialVelocityX(Vx);
      FInertia.put_InitialVelocityY(Vy);

      FInertia.put_InitialOriginX(X);
      FInertia.put_InitialOriginY(Y);

      FCompleted := False;
    end;
  end;
end;

function TGlowSpot.ManipulationCompleted(X, Y, cumulativeTranslationX,
  cumulativeTranslationY, cumulativeScale, cumulativeExpansion,
  cumulativeRotation: Single): HRESULT;
begin
  Result := S_OK;
end;

function TGlowSpot.ManipulationDelta(X, Y, translationDeltaX, translationDeltaY,
  scaleDelta, expansionDelta, rotationDelta, cumulativeTranslationX,
  cumulativeTranslationY, cumulativeScale, cumulativeExpansion,
  cumulativeRotation: Single): HRESULT;
begin
  Inc(Self.X, Round(translationDeltaX));
  Inc(Self.Y, Round(translationDeltaY));
  Result := S_OK;
end;

function TGlowSpot.ManipulationStarted(X, Y: Single): HRESULT;
begin
  Result := S_OK;
end;

procedure TGlowSpot.ProcessInertia;
begin
  if not FCompleted then
    FInertia.Process(FCompleted);
end;

The last bit of code I want to talk about is the mouse handling. I refactored the handling of the WM_TOUCH message into WMTouch and ProcessTouchMessages. ProcessTouchMessages is the common function that the mouse message can call to get the same inertia behavior.

procedure TTouchForm.WMTouch(var Message: TMessage);

  function TouchPointToPoint(const TouchPoint: TTouchInput): TPoint;
  begin
    Result := Point(TouchPoint.X div 100, TouchPoint.Y div 100);
    PhysicalToLogicalPoint(Handle, Result);
  end;

var
  TouchInputs: array of TTouchInput;
  TouchInput: TTouchInput;
  Handled: Boolean;
  Point: TPoint;
  TouchMessage: TTouchMessage;
begin
  Handled := False;
  SetLength(TouchInputs, Message.WParam);
  GetTouchInputInfo(Message.LParam, Message.WParam, @TouchInputs[0],
    SizeOf(TTouchInput));
  try
    for TouchInput in TouchInputs do
    begin
      Point := TouchPointToPoint(TouchInput);

      if (TouchInput.dwFlags and TOUCHEVENTF_MOVE) <> 0 then
        TouchMessage := tmMove
      else if (TouchInput.dwFlags and TOUCHEVENTF_UP) <> 0 then
        TouchMessage := tmUp
      else if (TouchInput.dwFlags and TOUCHEVENTF_DOWN) <> 0 then
        TouchMessage := tmDown;

      ProcessTouchMessages(Point, TouchInput.dwID, TouchMessage);
    end;

    Handled := True;
  finally
    if Handled then
      CloseTouchInputHandle(Message.LParam)
    else
      inherited;
  end;
end;

function TTouchForm.ProcessTouchMessages(const APoint: TPoint; ID: Integer;
  TouchMessage: TTouchMessage): TGlowSpot;
var
  Spot: TGlowSpot;
begin
  Result := nil;
  Spot := FindSpot(ID);

  if Spot = nil then
  begin
    Spot := FindSpot(APoint);

    if Spot <> nil then
      Spot.ID := ID;
  end;

  if Spot = nil then
  begin
    Spot := TGlowSpot.Create(Self);
    Spot.ID := ID;
    FSpots.Add(Spot);
  end;

  Spot.DoTouch(APoint, ID, TouchMessage);
  Result := Spot;
end;

And last the mouse handling functions with the check for ssTouch in the Shift parameter. If that check wasn't there each message would happen effectively twice because the touch messages also send mouse messages for backwards compatibility with applications that don't support touch.

procedure TTouchForm.FormMouseDown(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
begin
  if ssTouch in Shift then Exit;
  FMouseDown := True;
  ProcessTouchMessages(Point(X, Y), 0, tmDown);
end;

procedure TTouchForm.FormMouseMove(Sender: TObject; Shift: TShiftState; X,
  Y: Integer);
begin
  if ssTouch in Shift then Exit;
  if FMouseDown then
    ProcessTouchMessages(Point(X, Y), 0, tmMove);
end;

procedure TTouchForm.FormMouseUp(Sender: TObject; Button: TMouseButton;
  Shift: TShiftState; X, Y: Integer);
var
  Spot: TGlowSpot;
begin
  if ssTouch in Shift then Exit;
  Spot := ProcessTouchMessages(Point(X, Y), 0, tmUp);
  Spot.ID := -1;
  FMouseDown := False;
end;

I just realized that there is a bug in the program. It will run just fine but crash pretty bad if it runs too long. I guess I never demoed it very long before. So I'll leave debugging this crash as an exercise for the user.

Dell Latitude XT2 XFR Rugged Tablet

Dell just introduced the Latitude XT2 XFR; the rugged version of the XT2 multitouch tablet machine I've mentioned a few times before only with a name that is more of a mouth full. So for those of you waiting for a rugged machine before jumping into multitouch wait no longer. The short spec list is 12 inch multitouch display, 5lbs and 1.5 inches thick and mil spec badassness! It also comes with Vista. I guess Windows 7 isn't rugged certified? Just as well, first thing I do when I get a new machine is format it anyway. For all the awesomeness it'll set you back some serious coin. The base model is $3,600 but I loaded one up for just under $5k! I don't think I can justify one of these to Nick, but damn!

Dell SX2210T - Multitouch Monitors

The first standalone multi-touch monitor is the SX2210T. Looks pretty awesome for those that want to add a multi-touch monitor tho an existing computer.

Updated: Link goes to the product page (which is up now) rather than the shopping cart.

Sony Touchscreen Computer/TV

From the specs I think this Sony Vaio L beats pants off the HP Touchsmart. It's got glass, HDMI input, 1080p (I'm not sure if this is the computer resolution as well) Blue-ray, NVIDIA graphics and the monitor can be used as a TV without turning on the computer. I'm not sure what the other computer specs are, but the mouse does look horrible. It also appears that the DVD drive isn't a slot loading drive so that's a bit odd. Check out Sony website for more information.

Touch Demo Part III

Previously I posted Part I and Part II of the Touch Demo.

This time around we will add the glowing spots and the basic handling of the WM_TOUCH message. Add Generics.Collections to your uses and copy the following code to your unit:

type
  TGlowSpot = class
  public
    X, Y, Radius: Integer;
    Alpha: Extended;
    FadeIn: Boolean;
    Color: TColor;

  public
    constructor Create(AParent: TWinControl);

    procedure Paint(Canvas: TDirect2DCanvas);
  end;

  TGlowSpotList = class(TList<TGlowSpot>);
...
implementation

{$R *.dfm}

procedure PaintGlow(Canvas: TDirect2DCanvas; Alpha: Single;
  X, Y, Radius: Integer; Color: TColor);
var
  Stops: array[0 .. 1] of TD2D1GradientStop;
  Gradient: ID2D1GradientStopCollection;
  BrushProperties: TD2D1RadialGradientBrushProperties;
  RadialBrush: ID2D1RadialGradientBrush;
  Brush: TDirect2DBrush;
begin
  Stops[0].position := 0;
  Stops[0].Color := D2D1ColorF(Color, Alpha);
  Stops[1].position := 1;
  Stops[1].Color := D2D1ColorF(Color, 0);
  Canvas.RenderTarget.CreateGradientStopCollection
    (@Stops[0], Length(Stops), D2D1_GAMMA_2_2, D2D1_EXTEND_MODE_CLAMP,
    Gradient);
  BrushProperties.center := D2D1PointF(X, Y);
  BrushProperties.gradientOriginOffset.X := 0;
  BrushProperties.gradientOriginOffset.Y := 0;
  BrushProperties.radiusX := Radius;
  BrushProperties.radiusY := Radius;
  Canvas.RenderTarget.CreateRadialGradientBrush
    (BrushProperties, nil, Gradient, RadialBrush);
  Brush := TDirect2DBrush.Create(Canvas);
  Brush.Handle := RadialBrush;
  Canvas.Pen.Color := clNone;
  Canvas.Brush := Brush;
  Canvas.Ellipse(X - Radius, Y - Radius,
    X + Radius, Y + Radius);
end;

{ TGlowButton }

constructor TGlowSpot.Create(AParent: TWinControl);
begin
  inherited Create;
  Alpha := 1;
  Radius := 80;
  FadeIn := False;
  Randomize;
  Color := RGB(Random(255), Random(256), Random(256));
end;

procedure TGlowSpot.Paint(Canvas: TDirect2DCanvas);
begin
  PaintGlow(Canvas, Alpha, X, Y, Radius, Color);
end;

Add a local to hold a lists of TGlowSpots to your form then create it and free it:

  TTouchForm = class(TForm)
...  
  private
    FSpots: TGlowSpotList;
...
constructor TTouchForm.Create(AOwner: TComponent);
begin
  inherited;
  FCanvas := TDirect2DCanvas.Create(Handle);
  FSpots := TGlowSpotList.Create;
end;

destructor TTouchForm.Destroy;
begin
  FCanvas.Free;
  FSpots.Free;
  inherited;
end;

Update the Paint routine to paint the spots:

procedure TTouchForm.Paint;
var
  Spot: TGlowSpot;
begin
  Canvas.BeginDraw;

  try
    // Clear Background
    Canvas.RenderTarget.Clear(D2D1ColorF(clBlack));

    for Spot in FSpots do
      Spot.Paint(Canvas);

    // FPS
    Canvas.Font.Color := clWhite;
    Canvas.Brush.Color := clNone;
    Canvas.Font.Size := 14;
    Canvas.TextOut(10, 10, FloatToStrF(FFPS, ffFixed, 2, 2) + ' FPS');
  finally
    Canvas.EndDraw;
  end;
end;

Modify the Update method to fade the spots:

procedure TTouchForm.Update;
var
  Spot: TGlowSpot;
begin
  Inc(FFrames);

  if GetTickCount - FStartTime >= 1000 then
  begin
    FFPS := FFrames;
    FFrames := 0;
    FStartTime := GetTickCount;
  end;

  for Spot in FSpots do
  begin
    if Spot.FadeIn then
      Spot.Alpha := Spot.Alpha + 0.012
    else
      Spot.Alpha := Spot.Alpha - 0.012;

    if Spot.Alpha < 0.3 then
    begin
      Spot.FadeIn := True;
      Spot.Alpha := 0.4
    end
    else if Spot.Alpha > 1 then
      Spot.FadeIn := False;
  end;
end;

And last handle the WM_TOUCH message:

  TTouchForm = class(TForm)
...
  procedure WMTouch(var Message: TMessage); message WM_TOUCH;
...
procedure TTouchForm.WMTouch(var Message: TMessage);

  function TouchPointToPoint(const TouchPoint: TTouchInput): TPoint;
  begin
    Result := Point(TouchPoint.X div 100, TouchPoint.Y div 100);
    PhysicalToLogicalPoint(Handle, Result);
  end;

var
  TouchInputs: array of TTouchInput;
  TouchInput: TTouchInput;
  Handled: Boolean;
  Point: TPoint;
  Spot: TGlowSpot;
begin
  Handled := False;
  SetLength(TouchInputs, Message.WParam);
  GetTouchInputInfo(Message.LParam, Message.WParam, @TouchInputs[0],
    SizeOf(TTouchInput));
  try
    for TouchInput in TouchInputs do
    begin
      Point := TouchPointToPoint(TouchInput);
      Spot := TGlowSpot.Create(Self);
      Spot.X := Point.X;
      Spot.Y := Point.Y;
      FSpots.Add(Spot);
    end;

    Handled := True;
  finally
    if Handled then
      CloseTouchInputHandle(Message.LParam)
    else
      inherited;
  end;
end;

If you ran your app at this point you wouldn't get any touch messages. You need to call RegisterTouchWindow and UnregisterTouchWindow. I find it easiest to call UnregisterTouchWindow in the FormClose:

procedure TTouchForm.FormClose(Sender: TObject; var Action: TCloseAction);
begin
  UnregisterTouchWindow(Handle);
end;

and to call RegisterTouchWindow in the CreateWnd:

procedure TTouchForm.CreateWnd;
begin
  inherited;
  RegisterTouchWindow(Handle, 0);
end;

Now when you run your app and press the screen you'll get glowing spots wherever you touch the screen. Depending on your hardware 1, 2 or more at a time.

UPDATE: There is a memory leak in this program because I didn't want to delete any code for the next step.

UPDATE: Fixed HTML to make TGlowSpotList = class(TList) actually read TGlowSpotList(TList<TGlowSpot>);

Touch Demo Part II

Last week I posted Touch Demo Part I. This is the continuation to that series.

In Part I I demonstrated how to add the Direct2D Canvas to a form. This step is another foundation step that will add frames to your application by adding a timer.

At the moment the form looks like this:

type
  TTouchForm = class(TForm)
  private
    FCanvas: TDirect2DCanvas;
  protected
    procedure Paint; override;
    procedure Resize; override;
    procedure WMEraseBkgnd(var Message: TWMEraseBkgnd); message WM_ERASEBKGND;

  public
    constructor Create(AOwner: TComponent); override;
    destructor Destroy; override;

    property Canvas: TDirect2DCanvas read FCanvas
      write FCanvas;
  end;  

Add the following private fields to keep track of the frames:

    FFPS: Integer;
    FFrames: Integer;
    FStartTime: Cardinal;

and initialize them in the FormShow:

procedure TTouchForm.FormShow(Sender: TObject);
begin
  FFPS := 0;
  FFrames := 0;
  FStartTime := GetTickCount;
end;

Next update the Paint function:

procedure TTouchForm.Paint;
begin
  Canvas.BeginDraw;
  try
    // Clear Background
    Canvas.RenderTarget.Clear(D2D1ColorF(clBlack));

    // FPS
    Canvas.Font.Color := clWhite;
    Canvas.Brush.Color := clNone;
    Canvas.Font.Size := 14;
    Canvas.TextOut(10, 10, FloatToStrF(FFPS, ffFixed, 2, 2) + ' FPS');  finally
    Canvas.EndDraw;
  end;
end;

And then add a protected Update method that updates the frames:

procedure TTouchForm.Update;
begin
  Inc(FFrames);

  if GetTickCount - FStartTime >= 1000 then
  begin
    FFPS := FFrames;
    FFrames := 0;
    FStartTime := GetTickCount;
  end;
end;

Last, drop a TTimer on your form and set the interval to 10. Also set the form's Align to alClient and the BorderStyle to bsNone.

Next create the OnTimer event and simply call two functions, Update and Paint:

procedure TTouchForm.Timer1Timer(Sender: TObject);
begin
  Update;
  Paint;
end;

At this point your form should look like this:

type
  TTouchForm = class(TForm)
    Timer1: TTimer;
    procedure Timer1Timer(Sender: TObject);
    procedure FormShow(Sender: TObject);
  private
    FCanvas: TDirect2DCanvas;
    FFPS: Integer;
    FFrames: Integer;
    FStartTime: Cardinal;
  protected
    procedure Update;
    procedure Paint; override;
    procedure Resize; override;
    procedure WMEraseBkgnd(var Message: TWMEraseBkgnd); message WM_ERASEBKGND;

  public
    constructor Create(AOwner: TComponent); override;
    destructor Destroy; override;

    property Canvas: TDirect2DCanvas read FCanvas
      write FCanvas;
  end;  

HP Touchsmart

If you buy a Touchsmart be sure to spend the extra duckets to get the NVIDIA graphics card. It comes with an NVIDIA GeForce 9300M which is a pretty smokin' card and you'll be much happier.

Touch Demo Part I

Over the next few posts I am going to walk you through building one of the touch demos I used in my CodeRage 4 Session. This demo demonstrates using Direct2D and the WM_TOUCH message. Before we get started here is a brief explanation of these two technologies.

Since Windows Vista GDI has become a legacy drawing API supported with software rendering on top of Direct3D. Direct3D is great but it is very raw. This is where Direct2D shines for applications. Direct2D has all the performance benefits of Direct3D and adds support for bitmaps and text and support for GDI Interop all with a much simpler API. And Delphi 2010 has made it even simpler by wrapping it up with a TDirect2DCanvas that is very similar to TCanvas. Direct2D supports remote rendering through Remote Desktop which is very cool.

WM_TOUCH is the raw multi-touch message that provides various states of contact with the input digitizer for hot sizzling touch applications.

Now that we got that out of the way, create yourself a VCL Application. We want to start painting with Direct2D so add Direct2D and D2D1 to the uses and a TDirect2DCanvas property to your form like this:

  private
    FCanvas: TDirect2DCanvas;
  ...
  public
    constructor Create(AOwner: TComponent); override;
    destructor Destroy; override;

    property Canvas: TDirect2DCanvas read FCanvas
      write FCanvas;
  end;
...
constructor TTouchForm.Create(AOwner: TComponent);
begin
  inherited;
  FCanvas := TDirect2DCanvas.Create(Handle);
end;

destructor TTouchForm.Destroy;
begin
  FCanvas.Free;
  inherited;
end;

This creates a Direct2D surface (TDirect2DCanvas) that we can draw to instead of the traditional GDI surface (TCanvas).

Depending on the type of application you build you may experience some flicker. To avoid this handle the message WM_ERASEBKGND and returns true.

procedure TTouchForm.WMEraseBkgnd(
  var Message: TWMEraseBkgnd);
begin
  Message.Result := 1;
end;

Next override Paint and just draw a line:

procedure TTouchForm.Paint;
begin
  Canvas.BeginDraw;
  try
    Canvas.Pen.Color := clRed;
    Canvas.MoveTo(10, 10);
    Canvas.LineTo(40, 40);
  finally
    Canvas.EndDraw;
  end;
end;

If you run your application at this point and resize the form you will noticed that the line changes size. This is because the view port is resizing to scale to the window and your line will do the same. To prevent this override the Resize method and add the following:

procedure TTouchForm.Resize;
var
  HwndTarget: ID2D1HwndRenderTarget;
begin
  inherited;
  if Supports(Canvas.RenderTarget,
    ID2D1HwndRenderTarget, HwndTarget) then
    HwndTarget.Resize(D2D1SizeU(ClientWidth, ClientHeight));
  Invalidate;
end;

Now you have a form that renders everything drawn to the Canvas through a Direct2D surface.

UPDATE: Added Invalidate to Resize because when resizing the window larger it will force a redraw but smaller will not because typically a redraw isn't required.

Delphi 2010 - Touch and Pen added to TShiftState

While we were researching adding Touch and Gesturing to VCL we quickly realized it was important to detect what type of input produced the mouse messages. So we introduced some new elements to TShiftState: ssTouch and ssPen:

  TShiftState = set of (ssShift, ssAlt, ssCtrl,
    ssLeft, ssRight, ssMiddle, ssDouble, ssTouch, ssPen);

One of the key reasons this is so important is when handling WM_TOUCH messages the mouse messages are also fired. Chances are you only want to handle one of them. On a non-touch machine you want to handle the mouse. So this allows input to be filtered and handled independently.

Touch Hardware

One of the first questions people ask about multi-touch is what hardware to get.

Everything is supported just at different levels. First you have touch screens that emulate the mouse such as most Point-of-Sale monitors. We have a Dell POS and a Planar PT 19. These are only good for big buttons. Any sort of dragging your finger across the screen is lost by the device/driver. The Planar doesn't loose the mouse as bad as the Dell.

Then you have the old school tablet laptops. These add pen and sometimes support touch with the finger as well. The pen resolution is equal to or better than the mouse. The finger with these older systems is an afterthought. Lenovo IBM X61 is an example of such a system.

Next you have all-in-one systems such as the Asus EeeTop which has better drivers/hardware than the Point-of-Sale monitors but they are still just a touch screen mouse. I.E. push the monitor with your finger and you get a left mouse click. The Asus has a pretty decent touch screen but still only supports one touch point and it is just emulating the mouse. The processor on the Asus is really slow.

Now enters multi-touch which only Windows 7 really supports. Some Vista systems support multi-touch but just go with Windows 7, you'll be much happier.

Building on the tablet there are currently two shipping multi-touch laptops. One by Dell and one by HP. These make touch a forethought and add multi-touch. The HP supports 2 touch points and with an updated driver 4. The Dell supports 10 touch points. They are the exact same display so it must be the driver or other components that allow the extra touch points on the Dell. A Dell is currently sitting on my desk and it is the nicest of the systems. It feels like a laptop that I would buy for myself. The Dell is priced a bit high but it is a nice slimline laptop. The HP feels a bit cheap and has some flowery patterns on it.

Lastly you have the all-in-one multi-touch systems such as the HP TouchSmart which features a large display and two touch points. We have purchased two batches of these. The first batch is pretty bad. I'm pretty sure these are the ones currently selling on Woot. The second batch is a lot better but the Dell and HP laptops I mentioned above are better. Update: If you buy one of these be sure to get the NVIDIA graphics card.

RAD Studio 2010 - Gestures and Multi-Touch

One of the most exciting features in RAD Studio 2010 is Gestures and Multi-Touch (or what I like to call Interactive Gestures). I can't really show much here because it requires video, but if you've seen the videos or my Delphi Live! talk then you've seen some of what can be done.



Gestures takes full advantage of synthetic properties so you don't see much in the Object Inspector until a GestureManager is assigned. But Interactive Gestures and Tablet Options can be fully utilized without any setup, just check a few check boxes and add some code to the OnGesture event.

Gestures of all flavors can be added per control but different flavors of gestures are mutually exclusive. So you can have a panel with the Gestures and another panel with Interactive Gestures. It's pretty slick.

RAD Studio 2010 - Touch Keyboard

The Touch Keyboard (TTouchKeyboard) is a new component designed specifically for touch systems. It supports a pile of languages and multi-touch. You can see the multi-touch in action in the screen shot above where I was holding down shift and pressing G. The Touch Keyboard is supported on Windows 2000, XP, Vista and Windows 7, but only Windows 7 gets multi-touch.

The Touch Keyboard is designed to be embedded in your applications where you need input from the user. Notice there is no Windows key. After much thought we decided to take a different approach to the Touch Keyboard than Microsoft did with their on screen keyboard. Our focus was on touch systems without a physical keyboard, hence the name, where Microsoft emulated the hardware keyboard.

A lot of thought was given to the colors and the font used. Touch systems are probably more likely to be used in harsh light so there needed to be a lot of contrast between the keys and the background.