Advance Image and Video Processing Lab

Description
The Advance Image/Video Processing lab is specially designed for research purpose based on image and video processing and uses the Embedded GPU platform to demonstrates working with multi core environment. This setup introduces the learner with open source graphic libraries for various applications related to image and video processing. Click here for demo. 
 
Features
  • Lab focuses on getting started with graphic libraries for advance image and video processing etc.
  • Pre-requisite lab solution for learners looking for advance computer graphics, virtual reality, Artificial Intelligence, advance robotics, Machine learning, deep learning, mobile computing, gaming etc.
  • Useful for inference of Deep Learning Model
  • Can incorporate different types of cameras (stereo camera/thermal camera etc) in addition to USB camera as required by the research activity
  • Lab contains Advance GPU embedded platform from NVIDIA along with software tools, accessories and documentation

Advance Image/Video Processing Board Features

  • NVIDIA Pascal™ Architecture GPU
  • Hexa core CPU
  • 8 GB L128 bit DDR4 Memory
  • 32 GB eMMC 5.1 Flash Storage
  • Connectivity to 802.11ac Wi-Fi and Bluetooth-Enabled Devices
  • 10/100/1000BASE-T Ethernet
  • Power On/Off
  • Reset
  • Force Recovery
  • User-Defined
  • USB 3.0 Type A
  • USB 2.0 Micro AB (supports recovery and host mode)
  • HDMI
  • M.2 Key E
  • PCI-E x4
  • Full-Size SD
  • SATA Data and Power
  • GPIOs, I2C, I2S, SPI, CAN*
  • TTL UART
  • External 19V AC Adapter

 

Specifications

  • Advanced Image/Video Processing Embedded Platform comprises of dual-core NVIDIA Denver2 + quad-core ARM Cortex-A57, 256-core Pascal GPU, 8GB LPDDR4, 128-bit interface, 32GB eMMC, 4kp60 H.264/H.265 encoder & decoder, Dual ISPs (Image Signal Processors), 1.4 gigapixel/sec MIPI CSI camera ingest,
  • Ports and Peripherals includes: HDMI 2.0, 802.11a/b/g/n/ac 2×2 867Mbps WiFi, Bluetooth 4.1, USB3, USB2, 10/100/1000 BASE-T Ethernet, 12 lanes MIPI CSI 2.0, 2.5 Gb/sec per lane, PCIe gen 2.0, 1×4 + 1×1 or 2×1 + 1×2, SATA, SD card, dual CAN bus, UART, SPI, I2C, I2S, GPIOs etc.,  
  • 1TB USB hard disk with preloaded Linux Image
  • Eco System configured to work with Embedded GPU Board.
  • HDMI Screen (Optional)
  • USB Camera
  • USB hub
  • USB mouse
  • USB Keyboard
  • SSD Drive (Optional)

Experiments

The lab provides ready to work examples or experiments designed to enable the learner to quick start with working on this platform. These experiments are drafted as per the learning needs enabling the researcher or the learner to reduce the learning curve and get acquainted with the platform. The following list of experiments are offered along with the lab for an immediate start. The experiments are listed and classified as per the learning outcome requirements and is made available as a lab workbook document along with the source code. Learners can take help of these ready to start experiments and focus on their research topic thus making the best use of the platform.
The following is the list of different experimentation made available with the lab:
 
Getting Started With Computer Graphics – OpenGL
  1. Write a program to demonstrate generation of large number of slightly varying objects with bindless rendering.
  2. Write a program to demonstrate blooming effect on rendered surfaces making it glow.
  3. Write a program to demonstrate access to GL textures using both reading and writing to image.
  4. Write a program to demonstrate particle expansion by accessing vertex shaders in parallel.
  5. Write a program to demonstrate water simulation by using compute shaders.
  6. Write a program to demonstrate use of vertex shaders to animate particles and write back result into vertex buffer.
  7. Write a program to demonstrate use of high performance and quality approximation of anti-aliasing.
  8. Write a program to demonstrate High Dynamic Range (HDR) imaging.
  9. Write a program to implement instancing to tessellate objects in real time.
  10. Write a program to implement instancing to accelerate drawing of simillar objects
  11. Write a program to demonstrate multi-pass filtering for motion blur of fast moving objects.
  12. Write a program to demonstrate motion blur using 2D multi-pass filter.
  13. Write a program to demonstrate large number of drawcalls overhead using openGL extension.
  14. Write a program to implement openGLPSI(Pixel Shader Interlock) feature to blend decals.
  15. Write a program to implement Path rendenring extension to draw 2D line art.
  16. Write a program to use path rendering and animate live cursive writting.
  17. Write a program to implement HarfBuzz text shaping engine library for strings of unicode.
  18. Write a program to implement path rendering to draw text like spokes in a wheel with 3D effect.
  19. Write a program to implement conventional 3D graphic of Tiger using path rendering.
  20. Write a program to implement path rendering to wrap a artwork of Tiger with multiple paths and animate.
  21. Write a program to implement optimization techniques to rendering process to improve app level CPU GPU timings.
  22. Write a program to simulate a cloud of particles and render its shadow on model or floor object.
  23. Write a program to implement skinned meshes over bones in vertex shaders for smooth deformation.
  24. Write a program to demonstrate two methods of simulating soft shadows.
  25. Write a program to implement terrain engine by using hardware tessellation.
  26. Write a program to implement Terrain using Texture Array for high performance.
  27. Write a program to implement rendering of OIT (Order Independent Transparency) using weighted blending.

 

Getting Started With Computer Graphics – Vulkan

Basics
  1. Write a program to render colored triangle on the screen.
  2. Write a program to demonstrate the use of pipeline state objects (pso) in one single renderpass.
  3. Write a program to demonstrate the use of descriptor sets for passing data to shader stages.
  4. Write a program to demonstrate the use of Dynamic uniform buffers for rendering multiple objects with multiple matrices stored in a single uniform buffer object. 
  5. Write a program to demonstrate the use of (push constants) small shader block accessed outside of uniforms for fast updates.
  6. Write a program to demonstrate the use of Shader specialization constants to create multiple pipelines with different lighting paths from a single "uber" shader.
  7. Write a program to demonstrate texture loading including mip maps.
  8. Write a program to demonstrate the use of cube map textures.
  9. Write a program to demonstrate the use of texture arrays to display 2D textures.
  10. Write a program to generates a 3D texture.
  11. Write a program to to load Model and texture maps.
  12. Write a program to demonstrate the use of sub pass to implements a deferred rendering setup with a forward transparency pass
  13. Write a program to demonstrate the use of offsceen rendering to render mirror surface from the original image.
  14. Write a program to implement a simple CPU based particle system. 
  15. Write a program to demonstrate the use of stencil buffer and it's compare functionality for rendering a 3D model with dynamic outlines.
Advanced
  1. Write a program to aenders a scene made of multiple parts with different materials and textures (Scene rendering).
  2. Write a program to implements multisample anti-aliasing (MSAA) using a renderpass with multisampled attachments and resolve attachments that get resolved into the visible frame buffer.
  3. Write a program to implements a high dynamic range rendering pipeline using 16/32 bit floating point precision for all internal formats, textures and calculations, including a bloom pass, manual exposure and tone mapping.
  4. Write a program to rendering shadows for a directional light source. (Shadow mapping)
  5. Write a program to implement projective cascaded shadow mapping for directional light sources (Cascaded shadow mapping)
  6. Write a program to implement omni directional shadows using a dynamic cube map.
  7. Write a program to demonstrate how to generate a complete texture mip-chain using texture mapping at runtime instead of loading offline generated mip-maps from a texture file.
  8. Write a program to load and render an animated skinned 3D model.
  9. Write a program to capturing and saving an image after a scene has been rendered.
Performance
  1. Write a program to implement multi threaded command buffer generation.
  2. Write a program to implement instanced mesh rendering.
  3. Write a program to demonstrate the use of indirect draw commands.
  4. Write a program to demonstrate the use of occlusion query for visbility testing.
  5. Write a program to demonstrate the use of query pool objects to gather statistics from different stages of the pipeline
Physically based rendering
Physical based rendering as a lighting technique that achieves a more realistic and dynamic look by applying approximations of bidirectional reflectance distribution functions based on measured real-world material parameters and environment lighting.
  1. Write a program to demonstrate a basic specular BRDF implementation with solid materials and fixed light sources on a grid of objects with varying material parameters.
  2. Write a program to demonstrate physical based rendering with image based lighting
  3. Write a program to demonstrate physical based rendering with a textured object (metal/roughness workflow) with image based lighting
Deferred
  1. Write a program to demonstrate deferred shading with multiple render targets
  2. Write a program to demonstrate multi sampling with explicit resolve for deferred shading
  3. Write a program to demonstrate deferred shading with shadows from multiple light sources using geometry shader instancing
  4. Write a program to add ambient occlusion in screen space to a 3D scene.
Compute shader
  1. Write a program to demonstrate the use of a compute shader with different convolution kernels in realtime.
  2. Write a program to demonstrate attraction based compute shader particle system.
  3. Write a program to demonstrate compute shader N-body simulation using two passes and shared compute shader memory.
  4. Write a program to demonstrate simple GPU ray tracer with shadows and reflections using a compute shader.
  5. Write a program to demonstrate compute shader cloth simulation.
  6. Write a program to demonstrate compute shader culling and LOD using indirect rendering.
Geometry shader
  1. Write a program to demonstrate geometry shader (vertex normal debugging).
  2. Write a program to demonstrate viewport array with single pass rendering using geometry shaders.
Tessellation shader
  1. Write a program to demonstrate tessellation shader PN triangles.
  2. Write a program to demonstrate the use of tessellation to renders a terrain.
Headless
Examples that run one-time tasks and don't make use of visual output (no window system integration). These can be run in environments where no user interface is available (blog entry).
  1. Write a program to demonstrate minimal headless rendering
  2. Write a program to demonstrate minimal headless compute shader
User interface
  1. Write a program to demonstrate text overlay rendering on-top of an existing scene using a separate render pass.
  2. Write a program to demonstrate font rendering using signed distance fields.
  3. Write a program to generates and renders a complex user interface with multiple windows, controls and user interaction on top of a 3D scene. 
Effects
  1. Write a program to demonstrate the basics of fullscreen shader effects.
  2. Write a program to demonstrate bloom effect with fullscreen shader effects.
  3. Write a program to implement multiple texture mapping methods to simulate depth based on texture information (Normal mapping, parallax mapping, steep parallax mapping and parallax occlusion mapping).
  4. Write a program to demonstrate the use of a spherical material capture texture array defining environment lighting and reflection information to fake complex lighting.
Extensions
  1. Write a program to demonstrate the use of push descriptors apply the push constants concept to descriptor sets.
  2. Write a program to demonstrate the use of the VK_EXT_debug_marker extension to set debug markers, regions and to name Vulkan objects for advanced debugging in graphics debuggers like RenderDoc.
Misc
  1. Write a program to demonstrate the use of animated gears using multiple uniform buffers
  2. Write a program to render a Vulkan demo scene with logos and mascots.

 

Getting Started With Computer Vision - OpenCV

  1. Write a program to display Hello World.
  2. Write a program to Read image apply sobel filter and display output image.
  3. Write a program to Read image apply sobel filter and display output image.
  4. Write a program to Read Video file and display video.
  5. Write a program to Read Video file apply sobel filter and write xvid video file.
  6. Write a program to capture image from USB Webcam, apply sobel filter to it and write to image file.
  7. Write a program to capture video from USB Webcam, apply sobel filter to it and write to xvid video file.
  8. Write a program to perform basic operation like resize over a image.
  9. Write a program to perform Simple Canny Edge filter over an image.
  10. Write a program to perform Canny Edge filter using blur technique to get desired result.
  11. Write a program to perform simple feature detection using OrbFeatureDetector over a video.
  12. Write a program to perform optical flow over feature detection to track features and show the tracking over a video.
  13. Write a program to perform object detection by comparing unique points of a object to a video and find the object.
  14. Write a program to perform object detection and match unique descriptors of object with video and draw lines to show match.
  15. Write a program to perform object detection using matching with desired object and put a box around if the object is near to what is described.
  16. Write a program to perform Face detection using Cascade Classifier.
  17. Write a program to perform Face detection using Cascade Classifier with Histogram.
  18. Write a program to perform Face detection using alternate Cascade Classifier profile.
  19. Write a program to perform Background and foreground segmentation using CPU.
  20. Write a program to perform Background and foreground segmentation using GPU.
  21. Write a program to perform laplace point edge detection using USB Webcam capture.
  22. Write a program to perform Houghlines detection over a art image using both CPU and GPU.
  23. Write a program to perform grabcut segmentation over selected section.

 

High Performance Image/Video Processing - VisionWorks

  1. Write a program to detect feature points and track them using Lucas-Kanade method.
  2. Write a program to detect lines and circles using Hough Transform.
  3. Write a program for video stabilization using Lucas-Kanade method of feature tracking.
  4. Write a program to detect motion using Iterative Motion Estimation Algorithm.
  5. Write a program to take stereo input/video and do stereo matching to provide merged output.
  6. Write a program to do alpha blending between two images (use image mask) using interoperation between VisionWorks, OpenCV and NPP.
  7. Write a program to take video input from camera or video and display output using interoperation between VisionWorks and OpenGL.
  8. Write a program to take input video or camera input and display as it is to test I/O facilities and camera.
  9. Write a program to track any rigid object using optical flow method and object can be selected real time by draging cursor using mouse or touch.
  10. Write a program to estimate 3D feature/structure using VisionWorks SFM pipeline over 2D images/video.
 

CONTENTS :

Resources Quantity
Advance Image/Video Processing Board 1
19V Power Supply 1
USB hard disk with pre-loaded Eco System,USB Camera, Keyboard, Mouse 1
HDMI Screen (Optional) 1