3D fish tracking system - Revision history

Ismailalbayrak at 15:44, 2 October 2020

2020-10-02T15:44:54Z

Ismailalbayrak at 15:44, 2 October 2020

2020-10-02T15:44:23Z

Ismailalbayrak at 15:39, 2 October 2020

2020-10-02T15:39:41Z

Ismailalbayrak at 15:26, 30 September 2020

2020-09-30T15:26:17Z

Ismailalbayrak at 15:25, 30 September 2020

2020-09-30T15:25:03Z

Ismailalbayrak at 15:22, 30 September 2020

2020-09-30T15:22:17Z

Ismailalbayrak at 15:19, 30 September 2020

2020-09-30T15:19:37Z

Ismailalbayrak at 15:14, 30 September 2020

2020-09-30T15:14:09Z

Ismailalbayrak at 15:13, 30 September 2020

2020-09-30T15:13:32Z

Ismailalbayrak at 15:12, 30 September 2020

2020-09-30T15:12:34Z

← Older revision		Revision as of 15:44, 2 October 2020
Line 36:		Line 36:

	=Other information=		=Other information=
−	The total costs of the present system is approx. 40’000 USD=35’000 € including camera set-up and ~~recoding~~ software. For current costs of the equipment, we recommend to ask the corresponding supplier listed below. Note that a cheaper camera and lens set-up can significantly reduce the total cost of the system. The MATLAB-based 3D tracking code developed by VAW will be freely available.	+	The total costs of the present system is approx. 40’000 USD=35’000 € including camera set-up and recording software. For current costs of the equipment, we recommend to ask the corresponding supplier listed below. Note that a cheaper camera and lens set-up can significantly reduce the total cost of the system. The MATLAB-based 3D tracking code developed by VAW will be freely available.

@@ Line 25: / Line 25: @@
 The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 1). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 2a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 2b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 2c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
-An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 1). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Detert et al., 2018).
+An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 1). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Figure 3a, Detert et al., 2018).
-D fish tracking is based on the detection of moving fish in each frame and associating the detections corresponding to the same fish over time. These are done by using a background subtraction algorithm and a Karman filter in MATLAB (Detert et al., 2018). The primary results of motion-based tracking are tracks in a distorted and uncalibrated 2D image frame coordinate system for each camera. Figure 3a and c show the three detected fish and noises caused by reflections from the glass window and their 2D tracks over time. After undistorting such frames and stereo calibrating the cameras, the 2D fish tracks are transferred to a 3D metric-space according to their epipolar geometry based on the camera parameters derived from the calibration (Figure 4).
+D fish tracking is based on the detection of moving fish in each frame and associating the detections corresponding to the same fish over time. These are done by using a background subtraction algorithm and a Karman filter in MATLAB (Detert et al., 2018). The primary results of motion-based tracking are tracks in a distorted and uncalibrated 2D image frame coordinate system for each camera. Figure 3b and c show the three detected fish and noises caused by reflections from the glass window and their 2D tracks over time. After undistorting such frames and stereo calibrating the cameras, the 2D fish tracks are transferred to a 3D metric-space according to their epipolar geometry based on the camera parameters derived from the calibration (Figure 4).
 The etho-hydraulic tests were done for a flow depth of 90 cm, flume width of 150 cm, distance of 150 cm between the cameras and average flow velocities up to 0.7 m/s. Under such conditions, the 3D fish tracking system provided fish positions in 3D with an accuracy of about ±5 cm and 20 fps. The challenges for a successful implementation of the system are: assignment of individual fish to the tracks, constant illumination of the flow, camera distortion, air bubbles and suspended sediment and humid conditions for the cameras.

@@ Line 5: / Line 5: @@
 [[file:3d_fish_tracking_equipment.jpg|thumb|250px|Figure 2: (a) Camera (acA2000-50gmNIR, Basler) with lens (FE185C086HA-1, Fujifilm), (b) waterproof housing for the camera and lens (Autovimation), (c) high performance computer for camera recording and network switch for camera connection (source: VAW).]]
 [[file:3d_fish_tracking_3d_ouput.jpg|thumb|250px|Figure 3: (a) Stereo view of a camera pair, (b) three detected fish and noise, (c) 2D tracks of three fish (source: VAW).]]
-[[file:3d_fish_tracking_fish_tracks.jpg|thumb|250px|Figure 5: Top view of 3D tracks of three fish from an etho-hydraulic test of fish guidance structure with horizontal bars (source: VAW).]]
+[[file:3d_fish_tracking_fish_tracks.jpg|thumb|250px|Figure 4: Top view of 3D tracks of three fish from an etho-hydraulic test of fish guidance structure with horizontal bars (source: VAW).]]
 Date: 2018

@@ Line 27: / Line 27: @@
 An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 1). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Detert et al., 2018).
-D fish tracking is based on the detection of moving fish in each frame and associating the detections corresponding to the same fish over time. These are done by using a background subtraction algorithm and a Karman filter in MATLAB (Detert et al., 2018). The primary results of motion-based tracking are tracks in a distorted and uncalibrated 2D image frame coordinate system for each camera. Figure 14a and c show the three detected fish and noises caused by reflections from the glass window and their 2D tracks over time. After undistorting such frames and stereo calibrating the cameras, the 2D fish tracks are transferred to a 3D metric-space according to their epipolar geometry based on the camera parameters derived from the calibration (Figure 15).
+D fish tracking is based on the detection of moving fish in each frame and associating the detections corresponding to the same fish over time. These are done by using a background subtraction algorithm and a Karman filter in MATLAB (Detert et al., 2018). The primary results of motion-based tracking are tracks in a distorted and uncalibrated 2D image frame coordinate system for each camera. Figure 3a and c show the three detected fish and noises caused by reflections from the glass window and their 2D tracks over time. After undistorting such frames and stereo calibrating the cameras, the 2D fish tracks are transferred to a 3D metric-space according to their epipolar geometry based on the camera parameters derived from the calibration (Figure 4).
 The etho-hydraulic tests were done for a flow depth of 90 cm, flume width of 150 cm, distance of 150 cm between the cameras and average flow velocities up to 0.7 m/s. Under such conditions, the 3D fish tracking system provided fish positions in 3D with an accuracy of about ±5 cm and 20 fps. The challenges for a successful implementation of the system are: assignment of individual fish to the tracks, constant illumination of the flow, camera distortion, air bubbles and suspended sediment and humid conditions for the cameras.

@@ Line 25: / Line 25: @@
 The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 1). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 2a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 2b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 2c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
-An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 1). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Figure 14a, Detert et al., 2018).
+An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 1). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Detert et al., 2018).
 D fish tracking is based on the detection of moving fish in each frame and associating the detections corresponding to the same fish over time. These are done by using a background subtraction algorithm and a Karman filter in MATLAB (Detert et al., 2018). The primary results of motion-based tracking are tracks in a distorted and uncalibrated 2D image frame coordinate system for each camera. Figure 14a and c show the three detected fish and noises caused by reflections from the glass window and their 2D tracks over time. After undistorting such frames and stereo calibrating the cameras, the 2D fish tracks are transferred to a 3D metric-space according to their epipolar geometry based on the camera parameters derived from the calibration (Figure 15).

@@ Line 2: / Line 2: @@
 =Quick summary=
-[[file:3d_fish_tracking_installation1.jpg|thumb|250px|Figure 1: 3D fish tracking system (empty of water) installed in the etho-hydraulic flume at VAW of ETH Zurich (source: VAW)]]
+[[file:3d_fish_tracking_installation1.jpg|thumb|250px|Figure 1: 3D fish tracking system installed in the etho-hydraulic flume at VAW of ETH Zurich (source: VAW)]]
 [[file:3d_fish_tracking_equipment.jpg|thumb|250px|Figure 2: (a) Camera (acA2000-50gmNIR, Basler) with lens (FE185C086HA-1, Fujifilm), (b) waterproof housing for the camera and lens (Autovimation), (c) high performance computer for camera recording and network switch for camera connection (source: VAW).]]
 [[file:3d_fish_tracking_3d_ouput.jpg|thumb|250px|Figure 3: (a) Stereo view of a camera pair, (b) three detected fish and noise, (c) 2D tracks of three fish (source: VAW).]]

@@ Line 23: / Line 23: @@
 Within the scope of FIThydro, VAW investigates two types of fish guidance structures (FGS), namely with horizontal (Figure 1) and vertical curved bars. These FGSs are tested with six different fish species under various hydraulic conditions to evaluate their fish guidance efficiencies and to understand fish behaviour. To this end, the 3D fish tracking system is further developed and tested in these etho-hdyraulic (live-fish) investigations. The present system is similar to that currently used by the German Federal Institute for Hydraulic Engineering (BAW) in Karlsruhe together with the German Federal Institute of Hydrology (BfG, 2018; Detert et al., 2018).
-The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 2). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 13a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 13b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 13c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
+The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 1). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 2a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 2b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 2c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
 An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 12, right). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Figure 14a, Detert et al., 2018).

@@ Line 21: / Line 21: @@
 =Application=
-Within the scope of FIThydro, VAW investigates two types of fish guidance structures (FGS), namely with horizontal (Figure 12) and vertical curved bars. These FGSs are tested with six different fish species under various hydraulic conditions to evaluate their fish guidance efficiencies and to understand fish behaviour. To this end, the 3D fish tracking system is further developed and tested in these etho-hdyraulic (live-fish) investigations. The present system is similar to that currently used by the German Federal Institute for Hydraulic Engineering (BAW) in Karlsruhe together with the German Federal Institute of Hydrology (BfG, 2018; Detert et al., 2018).
+Within the scope of FIThydro, VAW investigates two types of fish guidance structures (FGS), namely with horizontal (Figure 1) and vertical curved bars. These FGSs are tested with six different fish species under various hydraulic conditions to evaluate their fish guidance efficiencies and to understand fish behaviour. To this end, the 3D fish tracking system is further developed and tested in these etho-hdyraulic (live-fish) investigations. The present system is similar to that currently used by the German Federal Institute for Hydraulic Engineering (BAW) in Karlsruhe together with the German Federal Institute of Hydrology (BfG, 2018; Detert et al., 2018).
-The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 13). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 13a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 13b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 13c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
+The present system consists of up to five cameras arranged in a streamwise series facing vertically upwards through the water surface, each with a distance of 1.5 m (Figure 2). Model acA2000-50gmNIR cameras from Basler are used and equipped with a 185° fisheye lens of FE185C086HA-1 (Fujifilm) (Figure 13a). The camera resolution is 3 MPx. Each camera and lens are waterproofed using a housing from Autovimation (Figure 13b). A GigE Vision 2.0 network with a Precision Time Protocol (PTP) IEEE1588 provided synchronous measurements with frame rates kept constant at 20 fps (Figure 13c). For larger control volume and longer areas, the actual system including the network switch and the high performance PC can theoretically be equipped with up to 48 cameras. However, the frame rate will be lower then.
 An adapted software by Fujifilm Switzerland is used to set-up cameras and record videos. The etho-hydraulic flume is illuminated with 7x1000 W halogen lamps (Figure 12, right). Calibration of the system is essential and made in three steps: finding intrinsic and extrinsic parameters for each of the five cameras using a checkboard, calibrating five stereo cameras according to the overlapping views of camera pairs, and finally performing a rigid transformation of all stereo camera pairs to a global flume coordinate system (Figure 14a, Detert et al., 2018).