Release 260308
This commit is contained in:
Comma Device
20
tools/CTF.md
Normal file
20
tools/CTF.md
Normal file
@@ -0,0 +1,20 @@
|
||||
## CTF
|
||||
Welcome to the first part of the comma CTF!
|
||||
|
||||
* all the flags are contained in this route: `0c7f0c7f0c7f0c7f|2021-10-13--13-00-00`
|
||||
* there's 2 flags in each segment, with roughly increasing difficulty
|
||||
* everything you'll need to find the flags is in the openpilot repo
|
||||
* grep is also your friend
|
||||
* first, [setup](https://github.com/commaai/openpilot/tree/master/tools#setup-your-pc) your PC
|
||||
* read the docs & checkout out the tools in tools/ and selfdrive/debug/
|
||||
* tip: once you get the replay and UI up, start by familiarizing yourself with seeking in replay
|
||||
|
||||
getting started
|
||||
```bash
|
||||
# start the route reply
|
||||
cd tools/replay
|
||||
./replay '0c7f0c7f0c7f0c7f|2021-10-13--13-00-00' --dcam --ecam
|
||||
|
||||
# start the UI in another terminal
|
||||
selfdrive/ui/ui
|
||||
```
|
||||
77
tools/README.md
Normal file
77
tools/README.md
Normal file
@@ -0,0 +1,77 @@
|
||||
# openpilot tools
|
||||
|
||||
## System Requirements
|
||||
|
||||
openpilot is developed and tested on **Ubuntu 20.04**, which is the primary development target aside from the [supported embedded hardware](https://github.com/commaai/openpilot#running-on-a-dedicated-device-in-a-car). We also have a CI test to verify that openpilot builds on macOS, but the tools are untested. For the best experience, stick to Ubuntu 20.04, otherwise openpilot and the tools should work with minimal to no modifications on macOS and other Linux systems.
|
||||
|
||||
## Setup your PC
|
||||
|
||||
First, clone openpilot:
|
||||
``` bash
|
||||
cd ~
|
||||
git clone --recurse-submodules https://github.com/commaai/openpilot.git
|
||||
|
||||
# or do a partial clone instead for a faster clone and smaller repo size
|
||||
git clone --filter=blob:none --recurse-submodules --also-filter-submodules https://github.com/commaai/openpilot.git
|
||||
|
||||
cd openpilot
|
||||
```
|
||||
|
||||
Then, run the setup script:
|
||||
|
||||
``` bash
|
||||
# for Ubuntu 20.04 LTS
|
||||
tools/ubuntu_setup.sh
|
||||
|
||||
# for macOS
|
||||
tools/mac_setup.sh
|
||||
```
|
||||
|
||||
Activate a shell with the Python dependencies installed:
|
||||
|
||||
``` bash
|
||||
cd openpilot && poetry shell
|
||||
```
|
||||
|
||||
Build openpilot with this command:
|
||||
``` bash
|
||||
scons -u -j$(nproc)
|
||||
```
|
||||
|
||||
### Dev Container
|
||||
|
||||
openpilot supports [Dev Containers](https://containers.dev/). Dev containers provide customizable and consistent development environment wrapped inside a container. This means you can develop in a designated environment matching our primary development target, regardless of your local setup.
|
||||
|
||||
Dev containers are supported in [multiple editors and IDEs](https://containers.dev/supporting), including [Visual Studio Code](https://code.visualstudio.com/docs/devcontainers/containers).
|
||||
|
||||
#### X11 forwarding on macOS
|
||||
|
||||
GUI apps like `ui` or `cabana` can also run inside the container by leveraging X11 forwarding. To make use of it on macOS, additional configuration steps must be taken. Follow [these](https://gist.github.com/sorny/969fe55d85c9b0035b0109a31cbcb088) steps to setup X11 forwarding on macOS.
|
||||
|
||||
### Windows
|
||||
|
||||
Neither openpilot nor any of the tools are developed or tested on Windows, but the [Windows Subsystem for Linux (WSL)](https://docs.microsoft.com/en-us/windows/wsl/about) should provide a similar experience to native Ubuntu. [WSL 2](https://docs.microsoft.com/en-us/windows/wsl/compare-versions) specifically has been reported by several users to be a seamless experience.
|
||||
|
||||
Follow [these instructions](https://docs.microsoft.com/en-us/windows/wsl/install) to setup the WSL and install the `Ubuntu-20.04` distribution. Once your Ubuntu WSL environment is setup, follow the Linux setup instructions to finish setting up your environment. See [these instructions](https://learn.microsoft.com/en-us/windows/wsl/tutorials/gui-apps) for running GUI apps.
|
||||
|
||||
**NOTE**: If you are running WSL and any GUIs are failing (segfaulting or other strange issues) even after following the steps above, you may need to enable software rendering with `LIBGL_ALWAYS_SOFTWARE=1`, e.g. `LIBGL_ALWAYS_SOFTWARE=1 selfdrive/ui/ui`.
|
||||
|
||||
## CTF
|
||||
Learn about the openpilot ecosystem and tools by playing our [CTF](/tools/CTF.md).
|
||||
|
||||
## Directory Structure
|
||||
|
||||
```
|
||||
├── ubuntu_setup.sh # Setup script for Ubuntu
|
||||
├── mac_setup.sh # Setup script for macOS
|
||||
├── cabana/ # View and plot CAN messages from drives or in realtime
|
||||
├── joystick/ # Control your car with a joystick
|
||||
├── lib/ # Libraries to support the tools and reading openpilot logs
|
||||
├── plotjuggler/ # A tool to plot openpilot logs
|
||||
├── replay/ # Replay drives and mock openpilot services
|
||||
├── scripts/ # Miscellaneous scripts
|
||||
├── serial/ # Tools for using the comma serial
|
||||
├── sim/ # Run openpilot in a simulator
|
||||
├── ssh/ # SSH into a comma device
|
||||
└── webcam/ # Run openpilot on a PC with webcams
|
||||
```
|
||||
0
tools/__init__.py
Normal file
0
tools/__init__.py
Normal file
159
tools/bodyteleop/bodyav.py
Normal file
159
tools/bodyteleop/bodyav.py
Normal file
@@ -0,0 +1,159 @@
|
||||
import asyncio
|
||||
import io
|
||||
import numpy as np
|
||||
import pyaudio
|
||||
import wave
|
||||
|
||||
from aiortc.contrib.media import MediaBlackhole
|
||||
from aiortc.mediastreams import AudioStreamTrack, MediaStreamError, MediaStreamTrack
|
||||
from aiortc.mediastreams import VIDEO_CLOCK_RATE, VIDEO_TIME_BASE
|
||||
from aiortc.rtcrtpsender import RTCRtpSender
|
||||
from av import CodecContext, Packet
|
||||
from pydub import AudioSegment
|
||||
import cereal.messaging as messaging
|
||||
|
||||
AUDIO_RATE = 16000
|
||||
SOUNDS = {
|
||||
'engage': '../../selfdrive/assets/sounds/engage.wav',
|
||||
'disengage': '../../selfdrive/assets/sounds/disengage.wav',
|
||||
'error': '../../selfdrive/assets/sounds/warning_immediate.wav',
|
||||
}
|
||||
|
||||
|
||||
def force_codec(pc, sender, forced_codec='video/VP9', stream_type="video"):
|
||||
codecs = RTCRtpSender.getCapabilities(stream_type).codecs
|
||||
codec = [codec for codec in codecs if codec.mimeType == forced_codec]
|
||||
transceiver = next(t for t in pc.getTransceivers() if t.sender == sender)
|
||||
transceiver.setCodecPreferences(codec)
|
||||
|
||||
|
||||
class EncodedBodyVideo(MediaStreamTrack):
|
||||
kind = "video"
|
||||
|
||||
_start: float
|
||||
_timestamp: int
|
||||
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
sock_name = 'livestreamDriverEncodeData'
|
||||
messaging.context = messaging.Context()
|
||||
self.sock = messaging.sub_sock(sock_name, None, conflate=True)
|
||||
self.pts = 0
|
||||
|
||||
async def recv(self) -> Packet:
|
||||
while True:
|
||||
msg = messaging.recv_one_or_none(self.sock)
|
||||
if msg is not None:
|
||||
break
|
||||
await asyncio.sleep(0.005)
|
||||
|
||||
evta = getattr(msg, msg.which())
|
||||
self.last_idx = evta.idx.encodeId
|
||||
|
||||
packet = Packet(evta.header + evta.data)
|
||||
packet.time_base = VIDEO_TIME_BASE
|
||||
packet.pts = self.pts
|
||||
self.pts += 0.05 * VIDEO_CLOCK_RATE
|
||||
return packet
|
||||
|
||||
|
||||
class WebClientSpeaker(MediaBlackhole):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
self.p = pyaudio.PyAudio()
|
||||
self.buffer = io.BytesIO()
|
||||
self.channels = 2
|
||||
self.stream = self.p.open(format=pyaudio.paInt16, channels=self.channels, rate=48000, frames_per_buffer=9600,
|
||||
output=True, stream_callback=self.pyaudio_callback)
|
||||
|
||||
def pyaudio_callback(self, in_data, frame_count, time_info, status):
|
||||
if self.buffer.getbuffer().nbytes < frame_count * self.channels * 2:
|
||||
buff = np.zeros((frame_count, 2), dtype=np.int16).tobytes()
|
||||
elif self.buffer.getbuffer().nbytes > 115200: # 3x the usual read size
|
||||
self.buffer.seek(0)
|
||||
buff = self.buffer.read(frame_count * self.channels * 4)
|
||||
buff = buff[:frame_count * self.channels * 2]
|
||||
self.buffer.seek(2)
|
||||
else:
|
||||
self.buffer.seek(0)
|
||||
buff = self.buffer.read(frame_count * self.channels * 2)
|
||||
self.buffer.seek(2)
|
||||
return (buff, pyaudio.paContinue)
|
||||
|
||||
async def consume(self, track):
|
||||
while True:
|
||||
try:
|
||||
frame = await track.recv()
|
||||
except MediaStreamError:
|
||||
return
|
||||
bio = bytes(frame.planes[0])
|
||||
self.buffer.write(bio)
|
||||
|
||||
async def start(self):
|
||||
for track, task in self._MediaBlackhole__tracks.items(): # pylint: disable=access-member-before-definition
|
||||
if task is None:
|
||||
self._MediaBlackhole__tracks[track] = asyncio.ensure_future(self.consume(track))
|
||||
|
||||
async def stop(self):
|
||||
for task in self._MediaBlackhole__tracks.values(): # pylint: disable=access-member-before-definition
|
||||
if task is not None:
|
||||
task.cancel()
|
||||
self._MediaBlackhole__tracks = {}
|
||||
self.stream.stop_stream()
|
||||
self.stream.close()
|
||||
self.p.terminate()
|
||||
|
||||
|
||||
class BodyMic(AudioStreamTrack):
|
||||
def __init__(self):
|
||||
super().__init__()
|
||||
|
||||
self.sample_rate = AUDIO_RATE
|
||||
self.AUDIO_PTIME = 0.020 # 20ms audio packetization
|
||||
self.samples = int(self.AUDIO_PTIME * self.sample_rate)
|
||||
self.FORMAT = pyaudio.paInt16
|
||||
self.CHANNELS = 2
|
||||
self.RATE = self.sample_rate
|
||||
self.CHUNK = int(AUDIO_RATE * 0.020)
|
||||
self.p = pyaudio.PyAudio()
|
||||
self.mic_stream = self.p.open(format=self.FORMAT, channels=1, rate=self.RATE, input=True, frames_per_buffer=self.CHUNK)
|
||||
|
||||
self.codec = CodecContext.create('pcm_s16le', 'r')
|
||||
self.codec.sample_rate = self.RATE
|
||||
self.codec.channels = 2
|
||||
self.audio_samples = 0
|
||||
self.chunk_number = 0
|
||||
|
||||
async def recv(self):
|
||||
mic_data = self.mic_stream.read(self.CHUNK)
|
||||
mic_sound = AudioSegment(mic_data, sample_width=2, channels=1, frame_rate=self.RATE)
|
||||
mic_sound = AudioSegment.from_mono_audiosegments(mic_sound, mic_sound)
|
||||
mic_sound += 3 # increase volume by 3db
|
||||
packet = Packet(mic_sound.raw_data)
|
||||
frame = self.codec.decode(packet)[0]
|
||||
frame.pts = self.audio_samples
|
||||
self.audio_samples += frame.samples
|
||||
self.chunk_number = self.chunk_number + 1
|
||||
return frame
|
||||
|
||||
|
||||
async def play_sound(sound):
|
||||
chunk = 5120
|
||||
with wave.open(SOUNDS[sound], 'rb') as wf:
|
||||
def callback(in_data, frame_count, time_info, status):
|
||||
data = wf.readframes(frame_count)
|
||||
return data, pyaudio.paContinue
|
||||
|
||||
p = pyaudio.PyAudio()
|
||||
stream = p.open(format=p.get_format_from_width(wf.getsampwidth()),
|
||||
channels=wf.getnchannels(),
|
||||
rate=wf.getframerate(),
|
||||
output=True,
|
||||
frames_per_buffer=chunk,
|
||||
stream_callback=callback)
|
||||
stream.start_stream()
|
||||
while stream.is_active():
|
||||
await asyncio.sleep(0)
|
||||
stream.stop_stream()
|
||||
stream.close()
|
||||
p.terminate()
|
||||
103
tools/bodyteleop/static/index.html
Normal file
103
tools/bodyteleop/static/index.html
Normal file
@@ -0,0 +1,103 @@
|
||||
<html>
|
||||
<head>
|
||||
<meta charset="UTF-8"/>
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>commabody</title>
|
||||
<link rel="stylesheet" href="/static/main.css">
|
||||
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/css/bootstrap.min.css" integrity="sha512-SbiR/eusphKoMVVXysTKG/7VseWii+Y3FdHrt0EpKgpToZeemhqHeZeLWLhJutz/2ut2Vw1uQEj2MbRF+TVBUA==" crossorigin="anonymous" referrerpolicy="no-referrer" /><script src="https://code.jquery.com/jquery-3.6.0.min.js"></script>
|
||||
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/bootstrap-icons@1.10.5/font/bootstrap-icons.css">
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/bootstrap/5.2.3/js/bootstrap.min.js" integrity="sha512-1/RvZTcCDEUjY/CypiMz+iqqtaoQfAITmNSJY17Myp4Ms5mdxPS5UV7iOfdZoxcGhzFbOm6sntTKJppjvuhg4g==" crossorigin="anonymous" referrerpolicy="no-referrer"></script>
|
||||
<script src="https://cdnjs.cloudflare.com/ajax/libs/moment.js/2.29.1/moment.min.js"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chart.js@^3"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/moment@^2"></script>
|
||||
<script src="https://cdn.jsdelivr.net/npm/chartjs-adapter-moment@^1"></script>
|
||||
</head>
|
||||
<body>
|
||||
<div id="main">
|
||||
<p class="jumbo">comma body</p>
|
||||
<audio id="audio" autoplay="true"></audio>
|
||||
<video id="video" playsinline autoplay muted loop poster="/static/poster.png"></video>
|
||||
<div id="icon-panel" class="row">
|
||||
<div class="col-sm-12 col-md-6 details">
|
||||
<div class="icon-sup-panel col-12">
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i class="bi bi-speaker-fill pre-blob"></i>
|
||||
<i class="bi bi-mic-fill pre-blob"></i>
|
||||
<i class="bi bi-camera-video-fill pre-blob"></i>
|
||||
</div>
|
||||
<p class="small">body</p>
|
||||
</div>
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i class="bi bi-speaker-fill pre-blob"></i>
|
||||
<i class="bi bi-mic-fill pre-blob"></i>
|
||||
</div>
|
||||
<p class="small">you</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-sm-12 col-md-6 details">
|
||||
<div class="icon-sup-panel col-12">
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i id="ping-time" class="pre-blob1">-</i>
|
||||
</div>
|
||||
<p class="bi bi-arrow-repeat small"> ping time</p>
|
||||
</div>
|
||||
<div class="icon-sub-panel">
|
||||
<div class="icon-sub-sub-panel">
|
||||
<i id="battery" class="pre-blob1">-</i>
|
||||
</div>
|
||||
<p class="bi bi-battery-half small"> battery</p>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<!-- <div class="icon-sub-panel">
|
||||
<button type="button" id="start" class="btn btn-light btn-lg">Start</button>
|
||||
<button type="button" id="stop" class="btn btn-light btn-lg">Stop</button>
|
||||
</div> -->
|
||||
</div>
|
||||
<div class="row" style="width: 100%; padding: 0px 10px 0px 10px;">
|
||||
<div id="wasd" class="col-md-12 row">
|
||||
<div class="col-md-6 col-sm-12" style="justify-content: center; display: flex; flex-direction: column;">
|
||||
<div class="wasd-row">
|
||||
<div class="keys" id="key-w">W</div>
|
||||
<div id="key-val"><span id="pos-vals">0,0</span><span>x,y</span></div>
|
||||
</div>
|
||||
<div class="wasd-row">
|
||||
<div class="keys" id="key-a">A</div>
|
||||
<div class="keys" id="key-s">S</div>
|
||||
<div class="keys" id="key-d">D</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="col-md-6 col-sm-12 form-group plan-form">
|
||||
<label for="plan-text">Plan (w, a, s, d, t)</label>
|
||||
<label style="font-size: 15px;" for="plan-text">*Extremely Experimental*</label>
|
||||
<textarea class="form-control" id="plan-text" rows="7" placeholder="1,0,0,0,2"></textarea>
|
||||
<button type="button" id="plan-button" class="btn btn-light btn-lg">Execute</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row" style="padding: 0px 10px 0px 10px; width: 100%;">
|
||||
<div class="panel row">
|
||||
<div class="col-sm-3" style="text-align: center;">
|
||||
<p>Play Sounds</p>
|
||||
</div>
|
||||
<div class="btn-group col-sm-8">
|
||||
<button type="button" id="sound-engage" class="btn btn-outline-success btn-lg sound">Engage</button>
|
||||
<button type="button" id="sound-disengage" class="btn btn-outline-warning btn-lg sound">Disengage</button>
|
||||
<button type="button" id="sound-error" class="btn btn-outline-danger btn-lg sound">Error</button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="row" style="padding: 0px 10px 0px 10px; width: 100%;">
|
||||
<div class="panel row">
|
||||
<div class="col-sm-6"><canvas id="chart-ping"></canvas></div>
|
||||
<div class="col-sm-6"><canvas id="chart-battery"></canvas></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<script src="/static/js/jsmain.js" type="module"></script>
|
||||
</body>
|
||||
</html>
|
||||
54
tools/bodyteleop/static/js/controls.js
vendored
Normal file
54
tools/bodyteleop/static/js/controls.js
vendored
Normal file
@@ -0,0 +1,54 @@
|
||||
const keyVals = {w: 0, a: 0, s: 0, d: 0}
|
||||
|
||||
export function getXY() {
|
||||
let x = -keyVals.w + keyVals.s
|
||||
let y = -keyVals.d + keyVals.a
|
||||
return {x, y}
|
||||
}
|
||||
|
||||
export const handleKeyX = (key, setValue) => {
|
||||
if (['w', 'a', 's', 'd'].includes(key)){
|
||||
keyVals[key] = setValue;
|
||||
let color = "#333";
|
||||
if (setValue === 1){
|
||||
color = "#e74c3c";
|
||||
}
|
||||
$("#key-"+key).css('background', color);
|
||||
const {x, y} = getXY();
|
||||
$("#pos-vals").text(x+","+y);
|
||||
}
|
||||
};
|
||||
|
||||
export async function executePlan() {
|
||||
let plan = $("#plan-text").val();
|
||||
const planList = [];
|
||||
plan.split("\n").forEach(function(e){
|
||||
let line = e.split(",").map(k=>parseInt(k));
|
||||
if (line.length != 5 || line.slice(0, 4).map(e=>[1, 0].includes(e)).includes(false) || line[4] < 0 || line[4] > 10){
|
||||
console.log("invalid plan");
|
||||
}
|
||||
else{
|
||||
planList.push(line)
|
||||
}
|
||||
});
|
||||
|
||||
async function execute() {
|
||||
for (var i = 0; i < planList.length; i++) {
|
||||
let [w, a, s, d, t] = planList[i];
|
||||
while(t > 0){
|
||||
console.log(w, a, s, d, t);
|
||||
if(w==1){$("#key-w").mousedown();}
|
||||
if(a==1){$("#key-a").mousedown();}
|
||||
if(s==1){$("#key-s").mousedown();}
|
||||
if(d==1){$("#key-d").mousedown();}
|
||||
await sleep(50);
|
||||
$("#key-w").mouseup();
|
||||
$("#key-a").mouseup();
|
||||
$("#key-s").mouseup();
|
||||
$("#key-d").mouseup();
|
||||
t = t - 0.05;
|
||||
}
|
||||
}
|
||||
}
|
||||
execute();
|
||||
}
|
||||
23
tools/bodyteleop/static/js/jsmain.js
Normal file
23
tools/bodyteleop/static/js/jsmain.js
Normal file
@@ -0,0 +1,23 @@
|
||||
import { handleKeyX, executePlan } from "./controls.js";
|
||||
import { start, stop, last_ping } from "./webrtc.js";
|
||||
|
||||
export var pc = null;
|
||||
export var dc = null;
|
||||
|
||||
document.addEventListener('keydown', (e)=>(handleKeyX(e.key.toLowerCase(), 1)));
|
||||
document.addEventListener('keyup', (e)=>(handleKeyX(e.key.toLowerCase(), 0)));
|
||||
$(".keys").bind("mousedown touchstart", (e)=>handleKeyX($(e.target).attr('id').replace('key-', ''), 1));
|
||||
$(".keys").bind("mouseup touchend", (e)=>handleKeyX($(e.target).attr('id').replace('key-', ''), 0));
|
||||
$("#plan-button").click(executePlan);
|
||||
|
||||
setInterval( () => {
|
||||
const dt = new Date().getTime();
|
||||
if ((dt - last_ping) > 1000) {
|
||||
$(".pre-blob").removeClass('blob');
|
||||
$("#battery").text("-");
|
||||
$("#ping-time").text('-');
|
||||
$("video")[0].load();
|
||||
}
|
||||
}, 5000);
|
||||
|
||||
start(pc, dc);
|
||||
53
tools/bodyteleop/static/js/plots.js
Normal file
53
tools/bodyteleop/static/js/plots.js
Normal file
@@ -0,0 +1,53 @@
|
||||
export const pingPoints = [];
|
||||
export const batteryPoints = [];
|
||||
|
||||
function getChartConfig(pts, color, title, ymax=100) {
|
||||
return {
|
||||
type: 'line',
|
||||
data: {
|
||||
datasets: [{
|
||||
label: title,
|
||||
data: pts,
|
||||
borderWidth: 1,
|
||||
borderColor: color,
|
||||
backgroundColor: color,
|
||||
fill: 'origin'
|
||||
}]
|
||||
},
|
||||
options: {
|
||||
scales: {
|
||||
x: {
|
||||
type: 'time',
|
||||
time: {
|
||||
unit: 'minute',
|
||||
displayFormats: {
|
||||
second: 'h:mm a'
|
||||
}
|
||||
},
|
||||
grid: {
|
||||
color: '#222', // Grid lines color
|
||||
},
|
||||
ticks: {
|
||||
source: 'data',
|
||||
fontColor: 'rgba(255, 255, 255, 1.0)', // Y-axis label color
|
||||
}
|
||||
},
|
||||
y: {
|
||||
beginAtZero: true,
|
||||
max: ymax,
|
||||
grid: {
|
||||
color: 'rgba(255, 255, 255, 0.1)', // Grid lines color
|
||||
},
|
||||
ticks: {
|
||||
fontColor: 'rgba(255, 255, 255, 0.7)', // Y-axis label color
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
const ctxPing = document.getElementById('chart-ping');
|
||||
const ctxBattery = document.getElementById('chart-battery');
|
||||
export const chartPing = new Chart(ctxPing, getChartConfig(pingPoints, 'rgba(192, 57, 43, 0.7)', 'Controls Ping Time (ms)', 250));
|
||||
export const chartBattery = new Chart(ctxBattery, getChartConfig(batteryPoints, 'rgba(41, 128, 185, 0.7)', 'Battery %', 100));
|
||||
217
tools/bodyteleop/static/js/webrtc.js
Normal file
217
tools/bodyteleop/static/js/webrtc.js
Normal file
@@ -0,0 +1,217 @@
|
||||
import { getXY } from "./controls.js";
|
||||
import { pingPoints, batteryPoints, chartPing, chartBattery } from "./plots.js";
|
||||
|
||||
export let dcInterval = null;
|
||||
export let batteryInterval = null;
|
||||
export let last_ping = null;
|
||||
|
||||
|
||||
export function createPeerConnection(pc) {
|
||||
var config = {
|
||||
sdpSemantics: 'unified-plan'
|
||||
};
|
||||
|
||||
pc = new RTCPeerConnection(config);
|
||||
|
||||
// connect audio / video
|
||||
pc.addEventListener('track', function(evt) {
|
||||
console.log("Adding Tracks!")
|
||||
if (evt.track.kind == 'video')
|
||||
document.getElementById('video').srcObject = evt.streams[0];
|
||||
else
|
||||
document.getElementById('audio').srcObject = evt.streams[0];
|
||||
});
|
||||
return pc;
|
||||
}
|
||||
|
||||
|
||||
export function negotiate(pc) {
|
||||
return pc.createOffer({offerToReceiveAudio:true, offerToReceiveVideo:true}).then(function(offer) {
|
||||
return pc.setLocalDescription(offer);
|
||||
}).then(function() {
|
||||
return new Promise(function(resolve) {
|
||||
if (pc.iceGatheringState === 'complete') {
|
||||
resolve();
|
||||
}
|
||||
else {
|
||||
function checkState() {
|
||||
if (pc.iceGatheringState === 'complete') {
|
||||
pc.removeEventListener('icegatheringstatechange', checkState);
|
||||
resolve();
|
||||
}
|
||||
}
|
||||
pc.addEventListener('icegatheringstatechange', checkState);
|
||||
}
|
||||
});
|
||||
}).then(function() {
|
||||
var offer = pc.localDescription;
|
||||
return fetch('/offer', {
|
||||
body: JSON.stringify({
|
||||
sdp: offer.sdp,
|
||||
type: offer.type,
|
||||
}),
|
||||
headers: {
|
||||
'Content-Type': 'application/json'
|
||||
},
|
||||
method: 'POST'
|
||||
});
|
||||
}).then(function(response) {
|
||||
console.log(response);
|
||||
return response.json();
|
||||
}).then(function(answer) {
|
||||
return pc.setRemoteDescription(answer);
|
||||
}).catch(function(e) {
|
||||
alert(e);
|
||||
});
|
||||
}
|
||||
|
||||
|
||||
function isMobile() {
|
||||
let check = false;
|
||||
(function(a){if(/(android|bb\d+|meego).+mobile|avantgo|bada\/|blackberry|blazer|compal|elaine|fennec|hiptop|iemobile|ip(hone|od)|iris|kindle|lge |maemo|midp|mmp|mobile.+firefox|netfront|opera m(ob|in)i|palm( os)?|phone|p(ixi|re)\/|plucker|pocket|psp|series(4|6)0|symbian|treo|up\.(browser|link)|vodafone|wap|windows ce|xda|xiino/i.test(a)||/1207|6310|6590|3gso|4thp|50[1-6]i|770s|802s|a wa|abac|ac(er|oo|s\-)|ai(ko|rn)|al(av|ca|co)|amoi|an(ex|ny|yw)|aptu|ar(ch|go)|as(te|us)|attw|au(di|\-m|r |s )|avan|be(ck|ll|nq)|bi(lb|rd)|bl(ac|az)|br(e|v)w|bumb|bw\-(n|u)|c55\/|capi|ccwa|cdm\-|cell|chtm|cldc|cmd\-|co(mp|nd)|craw|da(it|ll|ng)|dbte|dc\-s|devi|dica|dmob|do(c|p)o|ds(12|\-d)|el(49|ai)|em(l2|ul)|er(ic|k0)|esl8|ez([4-7]0|os|wa|ze)|fetc|fly(\-|_)|g1 u|g560|gene|gf\-5|g\-mo|go(\.w|od)|gr(ad|un)|haie|hcit|hd\-(m|p|t)|hei\-|hi(pt|ta)|hp( i|ip)|hs\-c|ht(c(\-| |_|a|g|p|s|t)|tp)|hu(aw|tc)|i\-(20|go|ma)|i230|iac( |\-|\/)|ibro|idea|ig01|ikom|im1k|inno|ipaq|iris|ja(t|v)a|jbro|jemu|jigs|kddi|keji|kgt( |\/)|klon|kpt |kwc\-|kyo(c|k)|le(no|xi)|lg( g|\/(k|l|u)|50|54|\-[a-w])|libw|lynx|m1\-w|m3ga|m50\/|ma(te|ui|xo)|mc(01|21|ca)|m\-cr|me(rc|ri)|mi(o8|oa|ts)|mmef|mo(01|02|bi|de|do|t(\-| |o|v)|zz)|mt(50|p1|v )|mwbp|mywa|n10[0-2]|n20[2-3]|n30(0|2)|n50(0|2|5)|n7(0(0|1)|10)|ne((c|m)\-|on|tf|wf|wg|wt)|nok(6|i)|nzph|o2im|op(ti|wv)|oran|owg1|p800|pan(a|d|t)|pdxg|pg(13|\-([1-8]|c))|phil|pire|pl(ay|uc)|pn\-2|po(ck|rt|se)|prox|psio|pt\-g|qa\-a|qc(07|12|21|32|60|\-[2-7]|i\-)|qtek|r380|r600|raks|rim9|ro(ve|zo)|s55\/|sa(ge|ma|mm|ms|ny|va)|sc(01|h\-|oo|p\-)|sdk\/|se(c(\-|0|1)|47|mc|nd|ri)|sgh\-|shar|sie(\-|m)|sk\-0|sl(45|id)|sm(al|ar|b3|it|t5)|so(ft|ny)|sp(01|h\-|v\-|v )|sy(01|mb)|t2(18|50)|t6(00|10|18)|ta(gt|lk)|tcl\-|tdg\-|tel(i|m)|tim\-|t\-mo|to(pl|sh)|ts(70|m\-|m3|m5)|tx\-9|up(\.b|g1|si)|utst|v400|v750|veri|vi(rg|te)|vk(40|5[0-3]|\-v)|vm40|voda|vulc|vx(52|53|60|61|70|80|81|83|85|98)|w3c(\-| )|webc|whit|wi(g |nc|nw)|wmlb|wonu|x700|yas\-|your|zeto|zte\-/i.test(a.substr(0,4))) check = true;})(navigator.userAgent||navigator.vendor||window.opera);
|
||||
return check;
|
||||
};
|
||||
|
||||
|
||||
export const constraints = {
|
||||
audio: {
|
||||
autoGainControl: false,
|
||||
sampleRate: 48000,
|
||||
sampleSize: 16,
|
||||
echoCancellation: true,
|
||||
noiseSuppression: true,
|
||||
channelCount: 1
|
||||
},
|
||||
video: isMobile()
|
||||
};
|
||||
|
||||
|
||||
export function createDummyVideoTrack() {
|
||||
const canvas = document.createElement('canvas');
|
||||
const context = canvas.getContext('2d');
|
||||
|
||||
const frameWidth = 5; // Set the width of the frame
|
||||
const frameHeight = 5; // Set the height of the frame
|
||||
canvas.width = frameWidth;
|
||||
canvas.height = frameHeight;
|
||||
|
||||
context.fillStyle = 'black';
|
||||
context.fillRect(0, 0, frameWidth, frameHeight);
|
||||
|
||||
const stream = canvas.captureStream();
|
||||
const videoTrack = stream.getVideoTracks()[0];
|
||||
|
||||
return videoTrack;
|
||||
}
|
||||
|
||||
|
||||
export function start(pc, dc) {
|
||||
pc = createPeerConnection(pc);
|
||||
|
||||
if (constraints.audio || constraints.video) {
|
||||
// add audio track
|
||||
navigator.mediaDevices.getUserMedia(constraints).then(function(stream) {
|
||||
stream.getTracks().forEach(function(track) {
|
||||
pc.addTrack(track, stream);
|
||||
// only audio?
|
||||
// if (track.kind === 'audio'){
|
||||
// pc.addTrack(track, stream);
|
||||
// }
|
||||
});
|
||||
return negotiate(pc);
|
||||
}, function(err) {
|
||||
alert('Could not acquire media: ' + err);
|
||||
});
|
||||
|
||||
// add a fake video?
|
||||
// const dummyVideoTrack = createDummyVideoTrack();
|
||||
// const dummyMediaStream = new MediaStream();
|
||||
// dummyMediaStream.addTrack(dummyVideoTrack);
|
||||
// pc.addTrack(dummyVideoTrack, dummyMediaStream);
|
||||
|
||||
} else {
|
||||
negotiate(pc);
|
||||
}
|
||||
|
||||
// setInterval(() => {pc.getStats(null).then((stats) => {stats.forEach((report) => console.log(report))})}, 10000)
|
||||
// var video = document.querySelector('video');
|
||||
// var print = function (e, f){console.log(e, f); video.requestVideoFrameCallback(print);};
|
||||
// video.requestVideoFrameCallback(print);
|
||||
|
||||
|
||||
var parameters = {"ordered": true};
|
||||
dc = pc.createDataChannel('data', parameters);
|
||||
dc.onclose = function() {
|
||||
console.log("data channel closed");
|
||||
clearInterval(dcInterval);
|
||||
clearInterval(batteryInterval);
|
||||
};
|
||||
function controlCommand() {
|
||||
const {x, y} = getXY();
|
||||
const dt = new Date().getTime();
|
||||
var message = JSON.stringify({type: 'control_command', x, y, dt});
|
||||
dc.send(message);
|
||||
}
|
||||
|
||||
function batteryLevel() {
|
||||
var message = JSON.stringify({type: 'battery_level'});
|
||||
dc.send(message);
|
||||
}
|
||||
|
||||
dc.onopen = function() {
|
||||
dcInterval = setInterval(controlCommand, 50);
|
||||
batteryInterval = setInterval(batteryLevel, 10000);
|
||||
controlCommand();
|
||||
batteryLevel();
|
||||
$(".sound").click((e)=>{
|
||||
const sound = $(e.target).attr('id').replace('sound-', '')
|
||||
dc.send(JSON.stringify({type: 'play_sound', sound}));
|
||||
});
|
||||
};
|
||||
|
||||
let val_print_idx = 0;
|
||||
dc.onmessage = function(evt) {
|
||||
const data = JSON.parse(evt.data);
|
||||
if(val_print_idx == 0 && data.type === 'ping_time') {
|
||||
const dt = new Date().getTime();
|
||||
const pingtime = dt - data.incoming_time;
|
||||
pingPoints.push({'x': dt, 'y': pingtime});
|
||||
if (pingPoints.length > 1000) {
|
||||
pingPoints.shift();
|
||||
}
|
||||
chartPing.update();
|
||||
$("#ping-time").text((pingtime) + "ms");
|
||||
last_ping = dt;
|
||||
$(".pre-blob").addClass('blob');
|
||||
}
|
||||
val_print_idx = (val_print_idx + 1 ) % 20;
|
||||
if(data.type === 'battery_level') {
|
||||
$("#battery").text(data.value + "%");
|
||||
batteryPoints.push({'x': new Date().getTime(), 'y': data.value});
|
||||
if (batteryPoints.length > 1000) {
|
||||
batteryPoints.shift();
|
||||
}
|
||||
chartBattery.update();
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
|
||||
export function stop(pc, dc) {
|
||||
if (dc) {
|
||||
dc.close();
|
||||
}
|
||||
if (pc.getTransceivers) {
|
||||
pc.getTransceivers().forEach(function(transceiver) {
|
||||
if (transceiver.stop) {
|
||||
transceiver.stop();
|
||||
}
|
||||
});
|
||||
}
|
||||
pc.getSenders().forEach(function(sender) {
|
||||
sender.track.stop();
|
||||
});
|
||||
setTimeout(function() {
|
||||
pc.close();
|
||||
}, 500);
|
||||
}
|
||||
185
tools/bodyteleop/static/main.css
Normal file
185
tools/bodyteleop/static/main.css
Normal file
@@ -0,0 +1,185 @@
|
||||
body {
|
||||
background: #333 !important;
|
||||
color: #fff !important;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: start;
|
||||
}
|
||||
|
||||
p {
|
||||
margin: 0px !important;
|
||||
}
|
||||
|
||||
i {
|
||||
font-style: normal;
|
||||
}
|
||||
|
||||
.small {
|
||||
font-size: 1em !important
|
||||
}
|
||||
|
||||
.jumbo {
|
||||
font-size: 8rem;
|
||||
}
|
||||
|
||||
|
||||
@media (max-width: 600px) {
|
||||
.small {
|
||||
font-size: 0.5em !important
|
||||
}
|
||||
.jumbo {
|
||||
display: none;
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
#main {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
align-content: center;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 30px;
|
||||
width: 100%;
|
||||
max-width: 1200px;
|
||||
}
|
||||
|
||||
video {
|
||||
width: 95%;
|
||||
}
|
||||
|
||||
.pre-blob {
|
||||
display: flex;
|
||||
background: #333;
|
||||
border-radius: 50%;
|
||||
margin: 10px;
|
||||
height: 45px;
|
||||
width: 45px;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
font-size: 1rem;
|
||||
}
|
||||
|
||||
.blob {
|
||||
background: rgba(231, 76, 60,1.0);
|
||||
box-shadow: 0 0 0 0 rgba(231, 76, 60,1.0);
|
||||
animation: pulse 2s infinite;
|
||||
}
|
||||
|
||||
@keyframes pulse {
|
||||
0% {
|
||||
box-shadow: 0 0 0 0px rgba(192, 57, 43, 1);
|
||||
}
|
||||
100% {
|
||||
box-shadow: 0 0 0 20px rgba(192, 57, 43, 0);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
.icon-sup-panel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: space-around;
|
||||
align-items: center;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
padding: 5px;
|
||||
margin: 5px 0px 5px 0px;
|
||||
}
|
||||
|
||||
.icon-sub-panel {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
#icon-panel {
|
||||
display: flex;
|
||||
width: 100%;
|
||||
justify-content: space-between;
|
||||
margin-top: 5px;
|
||||
}
|
||||
|
||||
.icon-sub-sub-panel {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
}
|
||||
|
||||
.keys, #key-val {
|
||||
background: #333;
|
||||
padding: 2rem;
|
||||
margin: 5px;
|
||||
color: #fff;
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
align-items: center;
|
||||
border-radius: 10px;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
#key-val {
|
||||
pointer-events: none;
|
||||
background: #fff;
|
||||
color: #333;
|
||||
line-height: 1;
|
||||
font-size: 20px;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.wasd-row {
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: center;
|
||||
align-items: stretch;
|
||||
}
|
||||
|
||||
#wasd {
|
||||
margin: 5px 0px 5px 0px;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
width: 100%;
|
||||
padding: 20px;
|
||||
display: flex;
|
||||
flex-direction: row;
|
||||
justify-content: space-around;
|
||||
align-items: stretch;
|
||||
|
||||
user-select: none;
|
||||
-webkit-touch-callout: none;
|
||||
-webkit-user-select: none;
|
||||
-khtml-user-select: none;
|
||||
-moz-user-select: none;
|
||||
-ms-user-select: none;
|
||||
touch-action: manipulation;
|
||||
}
|
||||
|
||||
.panel {
|
||||
display: flex;
|
||||
justify-content: center;
|
||||
margin: 5px 0px 5px 0px !important;
|
||||
background: #222;
|
||||
border-radius: 10px;
|
||||
width: 100%;
|
||||
padding: 10px;
|
||||
}
|
||||
|
||||
#ping-time, #battery {
|
||||
font-size: 25px;
|
||||
}
|
||||
|
||||
#stop {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.plan-form {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
}
|
||||
|
||||
.details {
|
||||
display: flex;
|
||||
padding: 0px 10px 0px 10px;
|
||||
}
|
||||
BIN
tools/bodyteleop/static/poster.png
Normal file
BIN
tools/bodyteleop/static/poster.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 8.1 KiB |
207
tools/bodyteleop/web.py
Normal file
207
tools/bodyteleop/web.py
Normal file
@@ -0,0 +1,207 @@
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import ssl
|
||||
import uuid
|
||||
import time
|
||||
|
||||
# aiortc and its dependencies have lots of internal warnings :(
|
||||
import warnings
|
||||
warnings.resetwarnings()
|
||||
warnings.simplefilter("always")
|
||||
|
||||
from aiohttp import web
|
||||
from aiortc import RTCPeerConnection, RTCSessionDescription
|
||||
|
||||
import cereal.messaging as messaging
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.tools.bodyteleop.bodyav import BodyMic, WebClientSpeaker, force_codec, play_sound, MediaBlackhole, EncodedBodyVideo
|
||||
|
||||
logger = logging.getLogger("pc")
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
|
||||
pcs = set()
|
||||
pm, sm = None, None
|
||||
TELEOPDIR = f"{BASEDIR}/tools/bodyteleop"
|
||||
|
||||
|
||||
async def index(request):
|
||||
content = open(TELEOPDIR + "/static/index.html", "r").read()
|
||||
now = time.monotonic()
|
||||
request.app['mutable_vals']['last_send_time'] = now
|
||||
request.app['mutable_vals']['last_override_time'] = now
|
||||
request.app['mutable_vals']['prev_command'] = []
|
||||
request.app['mutable_vals']['find_person'] = False
|
||||
|
||||
return web.Response(content_type="text/html", text=content)
|
||||
|
||||
|
||||
async def control_body(data, app):
|
||||
now = time.monotonic()
|
||||
if (data['type'] == 'dummy_controls') and (now < (app['mutable_vals']['last_send_time'] + 0.2)):
|
||||
return
|
||||
if (data['type'] == 'control_command') and (app['mutable_vals']['prev_command'] == [data['x'], data['y']] and data['x'] == 0 and data['y'] == 0):
|
||||
return
|
||||
|
||||
logger.info(str(data))
|
||||
x = max(-1.0, min(1.0, data['x']))
|
||||
y = max(-1.0, min(1.0, data['y']))
|
||||
dat = messaging.new_message('testJoystick')
|
||||
dat.testJoystick.axes = [x, y]
|
||||
dat.testJoystick.buttons = [False]
|
||||
pm.send('testJoystick', dat)
|
||||
app['mutable_vals']['last_send_time'] = now
|
||||
if (data['type'] == 'control_command'):
|
||||
app['mutable_vals']['last_override_time'] = now
|
||||
app['mutable_vals']['prev_command'] = [data['x'], data['y']]
|
||||
|
||||
|
||||
async def dummy_controls_msg(app):
|
||||
while True:
|
||||
if 'last_send_time' in app['mutable_vals']:
|
||||
this_time = time.monotonic()
|
||||
if (app['mutable_vals']['last_send_time'] + 0.2) < this_time:
|
||||
await control_body({'type': 'dummy_controls', 'x': 0, 'y': 0}, app)
|
||||
await asyncio.sleep(0.2)
|
||||
|
||||
|
||||
async def start_background_tasks(app):
|
||||
app['bgtask_dummy_controls_msg'] = asyncio.create_task(dummy_controls_msg(app))
|
||||
|
||||
|
||||
async def stop_background_tasks(app):
|
||||
app['bgtask_dummy_controls_msg'].cancel()
|
||||
await app['bgtask_dummy_controls_msg']
|
||||
|
||||
|
||||
async def offer(request):
|
||||
logger.info("\n\n\nnewoffer!\n\n")
|
||||
|
||||
params = await request.json()
|
||||
offer = RTCSessionDescription(sdp=params["sdp"], type=params["type"])
|
||||
speaker = WebClientSpeaker()
|
||||
blackhole = MediaBlackhole()
|
||||
|
||||
pc = RTCPeerConnection()
|
||||
pc_id = "PeerConnection(%s)" % uuid.uuid4()
|
||||
pcs.add(pc)
|
||||
|
||||
def log_info(msg, *args):
|
||||
logger.info(pc_id + " " + msg, *args)
|
||||
|
||||
log_info("Created for %s", request.remote)
|
||||
|
||||
@pc.on("datachannel")
|
||||
def on_datachannel(channel):
|
||||
request.app['mutable_vals']['remote_channel'] = channel
|
||||
|
||||
@channel.on("message")
|
||||
async def on_message(message):
|
||||
data = json.loads(message)
|
||||
if data['type'] == 'control_command':
|
||||
await control_body(data, request.app)
|
||||
times = {
|
||||
'type': 'ping_time',
|
||||
'incoming_time': data['dt'],
|
||||
'outgoing_time': int(time.time() * 1000),
|
||||
}
|
||||
channel.send(json.dumps(times))
|
||||
if data['type'] == 'battery_level':
|
||||
sm.update(timeout=0)
|
||||
if sm.updated['carState']:
|
||||
channel.send(json.dumps({'type': 'battery_level', 'value': int(sm['carState'].fuelGauge * 100)}))
|
||||
if data['type'] == 'play_sound':
|
||||
logger.info(f"Playing sound: {data['sound']}")
|
||||
await play_sound(data['sound'])
|
||||
if data['type'] == 'find_person':
|
||||
request.app['mutable_vals']['find_person'] = data['value']
|
||||
|
||||
@pc.on("connectionstatechange")
|
||||
async def on_connectionstatechange():
|
||||
log_info("Connection state is %s", pc.connectionState)
|
||||
if pc.connectionState == "failed":
|
||||
await pc.close()
|
||||
pcs.discard(pc)
|
||||
|
||||
@pc.on('track')
|
||||
def on_track(track):
|
||||
logger.info(f"Track received: {track.kind}")
|
||||
if track.kind == "audio":
|
||||
speaker.addTrack(track)
|
||||
elif track.kind == "video":
|
||||
blackhole.addTrack(track)
|
||||
|
||||
@track.on("ended")
|
||||
async def on_ended():
|
||||
log_info("Remote %s track ended", track.kind)
|
||||
if track.kind == "audio":
|
||||
await speaker.stop()
|
||||
elif track.kind == "video":
|
||||
await blackhole.stop()
|
||||
|
||||
video_sender = pc.addTrack(EncodedBodyVideo())
|
||||
force_codec(pc, video_sender, forced_codec='video/H264')
|
||||
_ = pc.addTrack(BodyMic())
|
||||
|
||||
await pc.setRemoteDescription(offer)
|
||||
await speaker.start()
|
||||
await blackhole.start()
|
||||
answer = await pc.createAnswer()
|
||||
await pc.setLocalDescription(answer)
|
||||
|
||||
return web.Response(
|
||||
content_type="application/json",
|
||||
text=json.dumps(
|
||||
{"sdp": pc.localDescription.sdp, "type": pc.localDescription.type}
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
async def on_shutdown(app):
|
||||
coros = [pc.close() for pc in pcs]
|
||||
await asyncio.gather(*coros)
|
||||
pcs.clear()
|
||||
|
||||
|
||||
async def run(cmd):
|
||||
proc = await asyncio.create_subprocess_shell(
|
||||
cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE
|
||||
)
|
||||
stdout, stderr = await proc.communicate()
|
||||
logger.info("Created key and cert!")
|
||||
if stdout:
|
||||
logger.info(f'[stdout]\n{stdout.decode()}')
|
||||
if stderr:
|
||||
logger.info(f'[stderr]\n{stderr.decode()}')
|
||||
|
||||
|
||||
def main():
|
||||
global pm, sm
|
||||
pm = messaging.PubMaster(['testJoystick'])
|
||||
sm = messaging.SubMaster(['carState', 'logMessage'])
|
||||
# App needs to be HTTPS for microphone and audio autoplay to work on the browser
|
||||
cert_path = TELEOPDIR + '/cert.pem'
|
||||
key_path = TELEOPDIR + '/key.pem'
|
||||
if (not os.path.exists(cert_path)) or (not os.path.exists(key_path)):
|
||||
asyncio.run(run(f'openssl req -x509 -newkey rsa:4096 -nodes -out {cert_path} -keyout {key_path} \
|
||||
-days 365 -subj "/C=US/ST=California/O=commaai/OU=comma body"'))
|
||||
else:
|
||||
logger.info("Certificate exists!")
|
||||
ssl_context = ssl.SSLContext()
|
||||
ssl_context.load_cert_chain(cert_path, key_path)
|
||||
app = web.Application()
|
||||
app['mutable_vals'] = {}
|
||||
app.on_shutdown.append(on_shutdown)
|
||||
app.router.add_post("/offer", offer)
|
||||
app.router.add_get("/", index)
|
||||
app.router.add_static('/static', TELEOPDIR + '/static')
|
||||
app.on_startup.append(start_background_tasks)
|
||||
app.on_cleanup.append(stop_background_tasks)
|
||||
web.run_app(app, access_log=None, host="0.0.0.0", port=5000, ssl_context=ssl_context)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
32
tools/cabana/README.md
Normal file
32
tools/cabana/README.md
Normal file
@@ -0,0 +1,32 @@
|
||||
# Cabana
|
||||
|
||||
Cabana is a tool developed to view raw CAN data. One use for this is creating and editing [CAN Dictionaries](http://socialledge.com/sjsu/index.php/DBC_Format) (DBC files), and the tool provides direct integration with [commaai/opendbc](https://github.com/commaai/opendbc) (a collection of DBC files), allowing you to load the DBC files direct from source, and save to your fork. In addition, you can load routes from [comma connect](https://connect.comma.ai).
|
||||
|
||||
## Usage Instructions
|
||||
|
||||
```bash
|
||||
$ ./cabana -h
|
||||
Usage: ./cabana [options] route
|
||||
|
||||
Options:
|
||||
-h, --help Displays help on commandline options.
|
||||
--help-all Displays help including Qt specific options.
|
||||
--demo use a demo route instead of providing your own
|
||||
--qcam load qcamera
|
||||
--ecam load wide road camera
|
||||
--stream read can messages from live streaming
|
||||
--panda read can messages from panda
|
||||
--panda-serial <panda-serial> read can messages from panda with given serial
|
||||
--socketcan <socketcan> read can messages from given SocketCAN device
|
||||
--zmq <zmq> the ip address on which to receive zmq
|
||||
messages
|
||||
--data_dir <data_dir> local directory with routes
|
||||
--no-vipc do not output video
|
||||
--dbc <dbc> dbc file to open
|
||||
|
||||
Arguments:
|
||||
route the drive to replay. find your drives at
|
||||
connect.comma.ai
|
||||
```
|
||||
|
||||
See [openpilot wiki](https://github.com/commaai/openpilot/wiki/Cabana)
|
||||
6
tools/cabana/assets/assets.qrc
Normal file
6
tools/cabana/assets/assets.qrc
Normal file
@@ -0,0 +1,6 @@
|
||||
<!DOCTYPE RCC><RCC version="1.0">
|
||||
<qresource>
|
||||
<file alias="bootstrap-icons.svg">../../../third_party/bootstrap/bootstrap-icons.svg</file>
|
||||
<file>cabana-icon.png</file>
|
||||
</qresource>
|
||||
</RCC>
|
||||
BIN
tools/cabana/assets/cabana-icon.png
Normal file
BIN
tools/cabana/assets/cabana-icon.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
90
tools/cabana/binaryview.h
Normal file
90
tools/cabana/binaryview.h
Normal file
@@ -0,0 +1,90 @@
|
||||
#pragma once
|
||||
|
||||
#include <tuple>
|
||||
#include <vector>
|
||||
|
||||
#include <QList>
|
||||
#include <QSet>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class BinaryItemDelegate : public QStyledItemDelegate {
|
||||
public:
|
||||
BinaryItemDelegate(QObject *parent);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
bool hasSignal(const QModelIndex &index, int dx, int dy, const cabana::Signal *sig) const;
|
||||
void drawSignalCell(QPainter* painter, const QStyleOptionViewItem &option, const QModelIndex &index, const cabana::Signal *sig) const;
|
||||
|
||||
QFont small_font, hex_font;
|
||||
};
|
||||
|
||||
class BinaryViewModel : public QAbstractTableModel {
|
||||
public:
|
||||
BinaryViewModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
void refresh();
|
||||
void updateState();
|
||||
void updateItem(int row, int col, const QString &val, const QColor &color);
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return row_count; }
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return column_count; }
|
||||
QModelIndex index(int row, int column, const QModelIndex &parent = QModelIndex()) const override {
|
||||
return createIndex(row, column, (void *)&items[row * column_count + column]);
|
||||
}
|
||||
Qt::ItemFlags flags(const QModelIndex &index) const override {
|
||||
return (index.column() == column_count - 1) ? Qt::ItemIsEnabled : Qt::ItemIsEnabled | Qt::ItemIsSelectable;
|
||||
}
|
||||
|
||||
struct Item {
|
||||
QColor bg_color = QColor(102, 86, 169, 255);
|
||||
bool is_msb = false;
|
||||
bool is_lsb = false;
|
||||
QString val;
|
||||
QList<const cabana::Signal *> sigs;
|
||||
bool valid = false;
|
||||
};
|
||||
std::vector<Item> items;
|
||||
|
||||
MessageId msg_id;
|
||||
int row_count = 0;
|
||||
const int column_count = 9;
|
||||
};
|
||||
|
||||
class BinaryView : public QTableView {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
BinaryView(QWidget *parent = nullptr);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void highlight(const cabana::Signal *sig);
|
||||
QSet<const cabana::Signal*> getOverlappingSignals() const;
|
||||
inline void updateState() { model->updateState(); }
|
||||
QSize minimumSizeHint() const override;
|
||||
|
||||
signals:
|
||||
void signalClicked(const cabana::Signal *sig);
|
||||
void signalHovered(const cabana::Signal *sig);
|
||||
void editSignal(const cabana::Signal *origin_s, cabana::Signal &s);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
|
||||
private:
|
||||
void addShortcuts();
|
||||
void refresh();
|
||||
std::tuple<int, int, bool> getSelection(QModelIndex index);
|
||||
void setSelection(const QRect &rect, QItemSelectionModel::SelectionFlags flags) override;
|
||||
void mousePressEvent(QMouseEvent *event) override;
|
||||
void mouseMoveEvent(QMouseEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
void leaveEvent(QEvent *event) override;
|
||||
void highlightPosition(const QPoint &pt);
|
||||
|
||||
QModelIndex anchor_index;
|
||||
BinaryViewModel *model;
|
||||
BinaryItemDelegate *delegate;
|
||||
const cabana::Signal *resize_sig = nullptr;
|
||||
const cabana::Signal *hovered_sig = nullptr;
|
||||
friend class BinaryItemDelegate;
|
||||
};
|
||||
114
tools/cabana/chart/chart.h
Normal file
114
tools/cabana/chart/chart.h
Normal file
@@ -0,0 +1,114 @@
|
||||
#pragma once
|
||||
|
||||
#include <tuple>
|
||||
#include <utility>
|
||||
|
||||
#include <QGraphicsPixmapItem>
|
||||
#include <QGraphicsProxyWidget>
|
||||
#include <QtCharts/QChartView>
|
||||
#include <QtCharts/QLegendMarker>
|
||||
#include <QtCharts/QLineSeries>
|
||||
#include <QtCharts/QScatterSeries>
|
||||
#include <QtCharts/QValueAxis>
|
||||
using namespace QtCharts;
|
||||
|
||||
#include "tools/cabana/chart/tiplabel.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
enum class SeriesType {
|
||||
Line = 0,
|
||||
StepLine,
|
||||
Scatter
|
||||
};
|
||||
|
||||
class ChartsWidget;
|
||||
class ChartView : public QChartView {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ChartView(const std::pair<double, double> &x_range, ChartsWidget *parent = nullptr);
|
||||
void addSignal(const MessageId &msg_id, const cabana::Signal *sig);
|
||||
bool hasSignal(const MessageId &msg_id, const cabana::Signal *sig) const;
|
||||
void updateSeries(const cabana::Signal *sig = nullptr, bool clear = true);
|
||||
void updatePlot(double cur, double min, double max);
|
||||
void setSeriesType(SeriesType type);
|
||||
void updatePlotArea(int left, bool force = false);
|
||||
void showTip(double sec);
|
||||
void hideTip();
|
||||
void startAnimation();
|
||||
|
||||
struct SigItem {
|
||||
MessageId msg_id;
|
||||
const cabana::Signal *sig = nullptr;
|
||||
QXYSeries *series = nullptr;
|
||||
QVector<QPointF> vals;
|
||||
QVector<QPointF> step_vals;
|
||||
uint64_t last_value_mono_time = 0;
|
||||
QPointF track_pt{};
|
||||
SegmentTree segment_tree;
|
||||
double min = 0;
|
||||
double max = 0;
|
||||
};
|
||||
|
||||
signals:
|
||||
void axisYLabelWidthChanged(int w);
|
||||
|
||||
private slots:
|
||||
void signalUpdated(const cabana::Signal *sig);
|
||||
void manageSignals();
|
||||
void handleMarkerClicked();
|
||||
void msgUpdated(MessageId id);
|
||||
void msgRemoved(MessageId id) { removeIf([=](auto &s) { return s.msg_id.address == id.address && !dbc()->msg(id); }); }
|
||||
void signalRemoved(const cabana::Signal *sig) { removeIf([=](auto &s) { return s.sig == sig; }); }
|
||||
|
||||
private:
|
||||
void createToolButtons();
|
||||
void addSeries(QXYSeries *series);
|
||||
void mousePressEvent(QMouseEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
void mouseMoveEvent(QMouseEvent *ev) override;
|
||||
void dragEnterEvent(QDragEnterEvent *event) override;
|
||||
void dragLeaveEvent(QDragLeaveEvent *event) override { drawDropIndicator(false); }
|
||||
void dragMoveEvent(QDragMoveEvent *event) override;
|
||||
void dropEvent(QDropEvent *event) override;
|
||||
void leaveEvent(QEvent *event) override;
|
||||
void resizeEvent(QResizeEvent *event) override;
|
||||
QSize sizeHint() const override;
|
||||
void updateAxisY();
|
||||
void updateTitle();
|
||||
void resetChartCache();
|
||||
void setTheme(QChart::ChartTheme theme);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void drawForeground(QPainter *painter, const QRectF &rect) override;
|
||||
void drawBackground(QPainter *painter, const QRectF &rect) override;
|
||||
void drawDropIndicator(bool draw) { if (std::exchange(can_drop, draw) != can_drop) viewport()->update(); }
|
||||
void drawTimeline(QPainter *painter);
|
||||
std::tuple<double, double, int> getNiceAxisNumbers(qreal min, qreal max, int tick_count);
|
||||
qreal niceNumber(qreal x, bool ceiling);
|
||||
QXYSeries *createSeries(SeriesType type, QColor color);
|
||||
void updateSeriesPoints();
|
||||
void removeIf(std::function<bool(const SigItem &)> predicate);
|
||||
inline void clearTrackPoints() { for (auto &s : sigs) s.track_pt = {}; }
|
||||
|
||||
int y_label_width = 0;
|
||||
int align_to = 0;
|
||||
QValueAxis *axis_x;
|
||||
QValueAxis *axis_y;
|
||||
QAction *split_chart_act;
|
||||
QGraphicsPixmapItem *move_icon;
|
||||
QGraphicsProxyWidget *close_btn_proxy;
|
||||
QGraphicsProxyWidget *manage_btn_proxy;
|
||||
TipLabel tip_label;
|
||||
QList<SigItem> sigs;
|
||||
double cur_sec = 0;
|
||||
SeriesType series_type = SeriesType::Line;
|
||||
bool is_scrubbing = false;
|
||||
bool resume_after_scrub = false;
|
||||
QPixmap chart_pixmap;
|
||||
bool can_drop = false;
|
||||
double tooltip_x = -1;
|
||||
QFont signal_value_font;
|
||||
ChartsWidget *charts_widget;
|
||||
friend class ChartsWidget;
|
||||
};
|
||||
129
tools/cabana/chart/chartswidget.h
Normal file
129
tools/cabana/chart/chartswidget.h
Normal file
@@ -0,0 +1,129 @@
|
||||
#pragma once
|
||||
|
||||
#include <unordered_map>
|
||||
#include <utility>
|
||||
|
||||
#include <QGridLayout>
|
||||
#include <QLabel>
|
||||
#include <QScrollArea>
|
||||
#include <QTimer>
|
||||
#include <QUndoCommand>
|
||||
#include <QUndoStack>
|
||||
|
||||
#include "tools/cabana/chart/signalselector.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
const int CHART_MIN_WIDTH = 300;
|
||||
const QString CHART_MIME_TYPE = "application/x-cabanachartview";
|
||||
|
||||
class ChartView;
|
||||
class ChartsWidget;
|
||||
|
||||
class ChartsContainer : public QWidget {
|
||||
public:
|
||||
ChartsContainer(ChartsWidget *parent);
|
||||
void dragEnterEvent(QDragEnterEvent *event) override;
|
||||
void dropEvent(QDropEvent *event) override;
|
||||
void dragLeaveEvent(QDragLeaveEvent *event) override { drawDropIndicator({}); }
|
||||
void drawDropIndicator(const QPoint &pt) { drop_indictor_pos = pt; update(); }
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
ChartView *getDropAfter(const QPoint &pos) const;
|
||||
|
||||
QGridLayout *charts_layout;
|
||||
ChartsWidget *charts_widget;
|
||||
QPoint drop_indictor_pos;
|
||||
};
|
||||
|
||||
class ChartsWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ChartsWidget(QWidget *parent = nullptr);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
inline bool hasSignal(const MessageId &id, const cabana::Signal *sig) { return findChart(id, sig) != nullptr; }
|
||||
|
||||
public slots:
|
||||
void setColumnCount(int n);
|
||||
void removeAll();
|
||||
void setZoom(double min, double max);
|
||||
|
||||
signals:
|
||||
void dock(bool floating);
|
||||
void rangeChanged(double min, double max, bool is_zommed);
|
||||
void seriesChanged();
|
||||
|
||||
private:
|
||||
void resizeEvent(QResizeEvent *event) override;
|
||||
bool event(QEvent *event) override;
|
||||
void alignCharts();
|
||||
void newChart();
|
||||
ChartView *createChart();
|
||||
void removeChart(ChartView *chart);
|
||||
void splitChart(ChartView *chart);
|
||||
QRect chartVisibleRect(ChartView *chart);
|
||||
void eventsMerged();
|
||||
void updateState();
|
||||
void zoomReset();
|
||||
void startAutoScroll();
|
||||
void stopAutoScroll();
|
||||
void doAutoScroll();
|
||||
void updateToolBar();
|
||||
void updateTabBar();
|
||||
void setMaxChartRange(int value);
|
||||
void updateLayout(bool force = false);
|
||||
void settingChanged();
|
||||
void showValueTip(double sec);
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
void newTab();
|
||||
void removeTab(int index);
|
||||
inline QList<ChartView *> ¤tCharts() { return tab_charts[tabbar->tabData(tabbar->currentIndex()).toInt()]; }
|
||||
ChartView *findChart(const MessageId &id, const cabana::Signal *sig);
|
||||
|
||||
QLabel *title_label;
|
||||
QLabel *range_lb;
|
||||
LogSlider *range_slider;
|
||||
QAction *range_lb_action;
|
||||
QAction *range_slider_action;
|
||||
bool docking = true;
|
||||
ToolButton *dock_btn;
|
||||
|
||||
QAction *undo_zoom_action;
|
||||
QAction *redo_zoom_action;
|
||||
QAction *reset_zoom_action;
|
||||
ToolButton *reset_zoom_btn;
|
||||
QUndoStack *zoom_undo_stack;
|
||||
|
||||
ToolButton *remove_all_btn;
|
||||
QList<ChartView *> charts;
|
||||
std::unordered_map<int, QList<ChartView *>> tab_charts;
|
||||
TabBar *tabbar;
|
||||
ChartsContainer *charts_container;
|
||||
QScrollArea *charts_scroll;
|
||||
uint32_t max_chart_range = 0;
|
||||
bool is_zoomed = false;
|
||||
std::pair<double, double> display_range;
|
||||
std::pair<double, double> zoomed_range;
|
||||
QAction *columns_action;
|
||||
int column_count = 1;
|
||||
int current_column_count = 0;
|
||||
int auto_scroll_count = 0;
|
||||
QTimer auto_scroll_timer;
|
||||
QTimer align_timer;
|
||||
int current_theme = 0;
|
||||
friend class ZoomCommand;
|
||||
friend class ChartView;
|
||||
friend class ChartsContainer;
|
||||
};
|
||||
|
||||
class ZoomCommand : public QUndoCommand {
|
||||
public:
|
||||
ZoomCommand(ChartsWidget *charts, std::pair<double, double> range) : charts(charts), range(range), QUndoCommand() {
|
||||
prev_range = charts->is_zoomed ? charts->zoomed_range : charts->display_range;
|
||||
setText(QObject::tr("Zoom to %1-%2").arg(range.first, 0, 'f', 1).arg(range.second, 0, 'f', 1));
|
||||
}
|
||||
void undo() override { charts->setZoom(prev_range.first, prev_range.second); }
|
||||
void redo() override { charts->setZoom(range.first, range.second); }
|
||||
ChartsWidget *charts;
|
||||
std::pair<double, double> prev_range, range;
|
||||
};
|
||||
30
tools/cabana/chart/signalselector.h
Normal file
30
tools/cabana/chart/signalselector.h
Normal file
@@ -0,0 +1,30 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QListWidget>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class SignalSelector : public QDialog {
|
||||
public:
|
||||
struct ListItem : public QListWidgetItem {
|
||||
ListItem(const MessageId &msg_id, const cabana::Signal *sig, QListWidget *parent) : msg_id(msg_id), sig(sig), QListWidgetItem(parent) {}
|
||||
MessageId msg_id;
|
||||
const cabana::Signal *sig;
|
||||
};
|
||||
|
||||
SignalSelector(QString title, QWidget *parent);
|
||||
QList<ListItem *> seletedItems();
|
||||
inline void addSelected(const MessageId &id, const cabana::Signal *sig) { addItemToList(selected_list, id, sig, true); }
|
||||
|
||||
private:
|
||||
void updateAvailableList(int index);
|
||||
void addItemToList(QListWidget *parent, const MessageId id, const cabana::Signal *sig, bool show_msg_name = false);
|
||||
void add(QListWidgetItem *item);
|
||||
void remove(QListWidgetItem *item);
|
||||
|
||||
QComboBox *msgs_combo;
|
||||
QListWidget *available_list;
|
||||
QListWidget *selected_list;
|
||||
};
|
||||
29
tools/cabana/chart/sparkline.h
Normal file
29
tools/cabana/chart/sparkline.h
Normal file
@@ -0,0 +1,29 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
|
||||
#include <QPixmap>
|
||||
#include <QPointF>
|
||||
#include <vector>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class Sparkline {
|
||||
public:
|
||||
void update(const MessageId &msg_id, const cabana::Signal *sig, double last_msg_ts, int range, QSize size);
|
||||
const QSize size() const { return pixmap.size() / pixmap.devicePixelRatio(); }
|
||||
inline double freq() const {
|
||||
return values.empty() ? 0 : values.size() / std::max(values.back().x() - values.front().x(), 1.0);
|
||||
}
|
||||
|
||||
QPixmap pixmap;
|
||||
double min_val = 0;
|
||||
double max_val = 0;
|
||||
double last_ts = 0;
|
||||
int time_range = 0;
|
||||
|
||||
private:
|
||||
void render(const QColor &color, QSize size);
|
||||
std::vector<QPointF> values;
|
||||
std::vector<QPointF> points;
|
||||
};
|
||||
10
tools/cabana/chart/tiplabel.h
Normal file
10
tools/cabana/chart/tiplabel.h
Normal file
@@ -0,0 +1,10 @@
|
||||
#pragma once
|
||||
|
||||
#include <QLabel>
|
||||
|
||||
class TipLabel : public QLabel {
|
||||
public:
|
||||
TipLabel(QWidget *parent = nullptr);
|
||||
void showText(const QPoint &pt, const QString &sec, QWidget *w, const QRect &rect);
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
};
|
||||
71
tools/cabana/commands.h
Normal file
71
tools/cabana/commands.h
Normal file
@@ -0,0 +1,71 @@
|
||||
#pragma once
|
||||
|
||||
#include <utility>
|
||||
|
||||
#include <QUndoCommand>
|
||||
#include <QUndoStack>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class EditMsgCommand : public QUndoCommand {
|
||||
public:
|
||||
EditMsgCommand(const MessageId &id, const QString &name, int size, const QString &comment, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QString old_name, new_name, old_comment, new_comment;
|
||||
int old_size = 0, new_size = 0;
|
||||
};
|
||||
|
||||
class RemoveMsgCommand : public QUndoCommand {
|
||||
public:
|
||||
RemoveMsgCommand(const MessageId &id, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
cabana::Msg message;
|
||||
};
|
||||
|
||||
class AddSigCommand : public QUndoCommand {
|
||||
public:
|
||||
AddSigCommand(const MessageId &id, const cabana::Signal &sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
bool msg_created = false;
|
||||
cabana::Signal signal = {};
|
||||
};
|
||||
|
||||
class RemoveSigCommand : public QUndoCommand {
|
||||
public:
|
||||
RemoveSigCommand(const MessageId &id, const cabana::Signal *sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QList<cabana::Signal> sigs;
|
||||
};
|
||||
|
||||
class EditSignalCommand : public QUndoCommand {
|
||||
public:
|
||||
EditSignalCommand(const MessageId &id, const cabana::Signal *sig, const cabana::Signal &new_sig, QUndoCommand *parent = nullptr);
|
||||
void undo() override;
|
||||
void redo() override;
|
||||
|
||||
private:
|
||||
const MessageId id;
|
||||
QList<std::pair<cabana::Signal, cabana::Signal>> sigs; // QList<{old_sig, new_sig}>
|
||||
};
|
||||
|
||||
namespace UndoStack {
|
||||
QUndoStack *instance();
|
||||
inline void push(QUndoCommand *cmd) { instance()->push(cmd); }
|
||||
};
|
||||
123
tools/cabana/dbc/dbc.h
Normal file
123
tools/cabana/dbc/dbc.h
Normal file
@@ -0,0 +1,123 @@
|
||||
#pragma once
|
||||
|
||||
#include <limits>
|
||||
#include <string>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QColor>
|
||||
#include <QList>
|
||||
#include <QMetaType>
|
||||
#include <QString>
|
||||
|
||||
#include "opendbc/can/common_dbc.h"
|
||||
|
||||
const QString UNTITLED = "untitled";
|
||||
|
||||
struct MessageId {
|
||||
uint8_t source = 0;
|
||||
uint32_t address = 0;
|
||||
|
||||
QString toString() const {
|
||||
return QString("%1:%2").arg(source).arg(address, 1, 16);
|
||||
}
|
||||
|
||||
bool operator==(const MessageId &other) const {
|
||||
return source == other.source && address == other.address;
|
||||
}
|
||||
|
||||
bool operator!=(const MessageId &other) const {
|
||||
return !(*this == other);
|
||||
}
|
||||
|
||||
bool operator<(const MessageId &other) const {
|
||||
return std::pair{source, address} < std::pair{other.source, other.address};
|
||||
}
|
||||
|
||||
bool operator>(const MessageId &other) const {
|
||||
return std::pair{source, address} > std::pair{other.source, other.address};
|
||||
}
|
||||
};
|
||||
|
||||
uint qHash(const MessageId &item);
|
||||
Q_DECLARE_METATYPE(MessageId);
|
||||
|
||||
template <>
|
||||
struct std::hash<MessageId> {
|
||||
std::size_t operator()(const MessageId &k) const noexcept { return qHash(k); }
|
||||
};
|
||||
|
||||
typedef QList<std::pair<double, QString>> ValueDescription;
|
||||
|
||||
namespace cabana {
|
||||
|
||||
class Signal {
|
||||
public:
|
||||
Signal() = default;
|
||||
Signal(const Signal &other) = default;
|
||||
void update();
|
||||
bool getValue(const uint8_t *data, size_t data_size, double *val) const;
|
||||
QString formatValue(double value) const;
|
||||
bool operator==(const cabana::Signal &other) const;
|
||||
inline bool operator!=(const cabana::Signal &other) const { return !(*this == other); }
|
||||
|
||||
enum class Type {
|
||||
Normal = 0,
|
||||
Multiplexed,
|
||||
Multiplexor
|
||||
};
|
||||
|
||||
Type type = Type::Normal;
|
||||
QString name;
|
||||
int start_bit, msb, lsb, size;
|
||||
double factor = 1.0;
|
||||
double offset = 0;
|
||||
bool is_signed;
|
||||
bool is_little_endian;
|
||||
double min, max;
|
||||
QString unit;
|
||||
QString comment;
|
||||
QString receiver_name;
|
||||
ValueDescription val_desc;
|
||||
int precision = 0;
|
||||
QColor color;
|
||||
|
||||
// Multiplexed
|
||||
int multiplex_value = 0;
|
||||
Signal *multiplexor = nullptr;
|
||||
};
|
||||
|
||||
class Msg {
|
||||
public:
|
||||
Msg() = default;
|
||||
Msg(const Msg &other) { *this = other; }
|
||||
~Msg();
|
||||
cabana::Signal *addSignal(const cabana::Signal &sig);
|
||||
cabana::Signal *updateSignal(const QString &sig_name, const cabana::Signal &sig);
|
||||
void removeSignal(const QString &sig_name);
|
||||
Msg &operator=(const Msg &other);
|
||||
int indexOf(const cabana::Signal *sig) const;
|
||||
cabana::Signal *sig(const QString &sig_name) const;
|
||||
QString newSignalName();
|
||||
void update();
|
||||
inline const std::vector<cabana::Signal *> &getSignals() const { return sigs; }
|
||||
|
||||
uint32_t address;
|
||||
QString name;
|
||||
uint32_t size;
|
||||
QString comment;
|
||||
QString transmitter;
|
||||
std::vector<cabana::Signal *> sigs;
|
||||
|
||||
std::vector<uint8_t> mask;
|
||||
cabana::Signal *multiplexor = nullptr;
|
||||
};
|
||||
|
||||
} // namespace cabana
|
||||
|
||||
// Helper functions
|
||||
double get_raw_value(const uint8_t *data, size_t data_size, const cabana::Signal &sig);
|
||||
void updateMsbLsb(cabana::Signal &s);
|
||||
inline int flipBitPos(int start_bit) { return 8 * (start_bit / 8) + 7 - start_bit % 8; }
|
||||
inline std::vector<std::string> allDBCNames() { return get_dbc_names(); }
|
||||
inline QString doubleToString(double value) { return QString::number(value, 'g', std::numeric_limits<double>::digits10); }
|
||||
44
tools/cabana/dbc/dbcfile.h
Normal file
44
tools/cabana/dbc/dbcfile.h
Normal file
@@ -0,0 +1,44 @@
|
||||
#pragma once
|
||||
|
||||
#include <map>
|
||||
#include <QObject>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
|
||||
const QString AUTO_SAVE_EXTENSION = ".tmp";
|
||||
|
||||
class DBCFile : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DBCFile(const QString &dbc_file_name, QObject *parent=nullptr);
|
||||
DBCFile(const QString &name, const QString &content, QObject *parent=nullptr);
|
||||
~DBCFile() {}
|
||||
|
||||
bool save();
|
||||
bool saveAs(const QString &new_filename);
|
||||
bool autoSave();
|
||||
bool writeContents(const QString &fn);
|
||||
void cleanupAutoSaveFile();
|
||||
QString generateDBC();
|
||||
|
||||
void updateMsg(const MessageId &id, const QString &name, uint32_t size, const QString &comment);
|
||||
inline void removeMsg(const MessageId &id) { msgs.erase(id.address); }
|
||||
|
||||
inline const std::map<uint32_t, cabana::Msg> &getMessages() const { return msgs; }
|
||||
cabana::Msg *msg(uint32_t address);
|
||||
cabana::Msg *msg(const QString &name);
|
||||
inline cabana::Msg *msg(const MessageId &id) { return msg(id.address); }
|
||||
|
||||
int signalCount();
|
||||
inline int msgCount() { return msgs.size(); }
|
||||
inline QString name() { return name_.isEmpty() ? "untitled" : name_; }
|
||||
inline bool isEmpty() { return (signalCount() == 0) && name_.isEmpty(); }
|
||||
|
||||
QString filename;
|
||||
|
||||
private:
|
||||
void parse(const QString &content);
|
||||
std::map<uint32_t, cabana::Msg> msgs;
|
||||
QString name_;
|
||||
};
|
||||
80
tools/cabana/dbc/dbcmanager.h
Normal file
80
tools/cabana/dbc/dbcmanager.h
Normal file
@@ -0,0 +1,80 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <map>
|
||||
#include <set>
|
||||
#include <vector>
|
||||
|
||||
#include "tools/cabana/dbc/dbcfile.h"
|
||||
|
||||
typedef std::set<int> SourceSet;
|
||||
const SourceSet SOURCE_ALL = {-1};
|
||||
const int INVALID_SOURCE = 0xff;
|
||||
inline bool operator<(const std::shared_ptr<DBCFile> &l, const std::shared_ptr<DBCFile> &r) { return l.get() < r.get(); }
|
||||
|
||||
class DBCManager : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DBCManager(QObject *parent) : QObject(parent) {}
|
||||
~DBCManager() {}
|
||||
bool open(const SourceSet &sources, const QString &dbc_file_name, QString *error = nullptr);
|
||||
bool open(const SourceSet &sources, const QString &name, const QString &content, QString *error = nullptr);
|
||||
void close(const SourceSet &sources);
|
||||
void close(DBCFile *dbc_file);
|
||||
void closeAll();
|
||||
|
||||
void addSignal(const MessageId &id, const cabana::Signal &sig);
|
||||
void updateSignal(const MessageId &id, const QString &sig_name, const cabana::Signal &sig);
|
||||
void removeSignal(const MessageId &id, const QString &sig_name);
|
||||
|
||||
void updateMsg(const MessageId &id, const QString &name, uint32_t size, const QString &comment);
|
||||
void removeMsg(const MessageId &id);
|
||||
|
||||
QString newMsgName(const MessageId &id);
|
||||
QString newSignalName(const MessageId &id);
|
||||
const std::vector<uint8_t>& mask(const MessageId &id);
|
||||
|
||||
const std::map<uint32_t, cabana::Msg> &getMessages(uint8_t source);
|
||||
cabana::Msg *msg(const MessageId &id);
|
||||
cabana::Msg* msg(uint8_t source, const QString &name);
|
||||
|
||||
QStringList signalNames();
|
||||
int signalCount(const MessageId &id);
|
||||
int signalCount();
|
||||
int msgCount();
|
||||
int dbcCount();
|
||||
int nonEmptyDBCCount();
|
||||
|
||||
const SourceSet sources(const DBCFile *dbc_file) const;
|
||||
DBCFile *findDBCFile(const uint8_t source);
|
||||
inline DBCFile *findDBCFile(const MessageId &id) { return findDBCFile(id.source); }
|
||||
std::set<DBCFile *> allDBCFiles();
|
||||
|
||||
signals:
|
||||
void signalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void signalRemoved(const cabana::Signal *sig);
|
||||
void signalUpdated(const cabana::Signal *sig);
|
||||
void msgUpdated(MessageId id);
|
||||
void msgRemoved(MessageId id);
|
||||
void DBCFileChanged();
|
||||
void maskUpdated();
|
||||
|
||||
private:
|
||||
std::map<int, std::shared_ptr<DBCFile>> dbc_files;
|
||||
};
|
||||
|
||||
DBCManager *dbc();
|
||||
|
||||
inline QString msgName(const MessageId &id) {
|
||||
auto msg = dbc()->msg(id);
|
||||
return msg ? msg->name : UNTITLED;
|
||||
}
|
||||
|
||||
inline QString toString(const SourceSet &ss) {
|
||||
QStringList ret;
|
||||
for (auto s : ss) {
|
||||
ret << (s == -1 ? QString("all") : QString::number(s));
|
||||
}
|
||||
return ret.join(", ");
|
||||
}
|
||||
24
tools/cabana/dbc/generate_dbc_json.py
Executable file
24
tools/cabana/dbc/generate_dbc_json.py
Executable file
@@ -0,0 +1,24 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
|
||||
from openpilot.selfdrive.car.car_helpers import get_interface_attr
|
||||
|
||||
|
||||
def generate_dbc_json() -> str:
|
||||
all_cars_by_brand = get_interface_attr("CAR_INFO")
|
||||
all_dbcs_by_brand = get_interface_attr("DBC")
|
||||
dbc_map = {car: all_dbcs_by_brand[brand][car]['pt'] for brand, cars in all_cars_by_brand.items() for car in cars if car != 'mock'}
|
||||
return json.dumps(dict(sorted(dbc_map.items())), indent=2)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Generate mapping for all car fingerprints to DBC names and outputs json file",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("--out", required=True, help="Generated json filepath")
|
||||
args = parser.parse_args()
|
||||
|
||||
with open(args.out, 'w') as f:
|
||||
f.write(generate_dbc_json())
|
||||
print(f"Generated and written to {args.out}")
|
||||
68
tools/cabana/detailwidget.h
Normal file
68
tools/cabana/detailwidget.h
Normal file
@@ -0,0 +1,68 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDialogButtonBox>
|
||||
#include <QSplitter>
|
||||
#include <QTabWidget>
|
||||
#include <QTextEdit>
|
||||
|
||||
#include "selfdrive/ui/qt/widgets/controls.h"
|
||||
#include "tools/cabana/binaryview.h"
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/historylog.h"
|
||||
#include "tools/cabana/signalview.h"
|
||||
|
||||
class MainWindow;
|
||||
class EditMessageDialog : public QDialog {
|
||||
public:
|
||||
EditMessageDialog(const MessageId &msg_id, const QString &title, int size, QWidget *parent);
|
||||
void validateName(const QString &text);
|
||||
|
||||
MessageId msg_id;
|
||||
QString original_name;
|
||||
QDialogButtonBox *btn_box;
|
||||
QLineEdit *name_edit;
|
||||
QTextEdit *comment_edit;
|
||||
QLabel *error_label;
|
||||
QSpinBox *size_spin;
|
||||
};
|
||||
|
||||
class DetailWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
DetailWidget(ChartsWidget *charts, QWidget *parent);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void refresh();
|
||||
|
||||
private:
|
||||
void showTabBarContextMenu(const QPoint &pt);
|
||||
void editMsg();
|
||||
void removeMsg();
|
||||
void updateState(const QHash<MessageId, CanData> * msgs = nullptr);
|
||||
|
||||
MessageId msg_id;
|
||||
QLabel *time_label, *warning_icon, *warning_label;
|
||||
ElidedLabel *name_label;
|
||||
QWidget *warning_widget;
|
||||
TabBar *tabbar;
|
||||
QTabWidget *tab_widget;
|
||||
QToolButton *remove_btn;
|
||||
LogsWidget *history_log;
|
||||
BinaryView *binary_view;
|
||||
SignalView *signal_view;
|
||||
ChartsWidget *charts;
|
||||
QSplitter *splitter;
|
||||
};
|
||||
|
||||
class CenterWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
CenterWidget(QWidget *parent);
|
||||
void setMessage(const MessageId &msg_id);
|
||||
void clear();
|
||||
|
||||
private:
|
||||
QWidget *createWelcomeWidget();
|
||||
DetailWidget *detail_widget = nullptr;
|
||||
QWidget *welcome_widget = nullptr;
|
||||
};
|
||||
94
tools/cabana/historylog.h
Normal file
94
tools/cabana/historylog.h
Normal file
@@ -0,0 +1,94 @@
|
||||
#pragma once
|
||||
|
||||
#include <deque>
|
||||
#include <vector>
|
||||
|
||||
#include <QCheckBox>
|
||||
#include <QComboBox>
|
||||
#include <QHeaderView>
|
||||
#include <QLineEdit>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
#include "tools/cabana/util.h"
|
||||
|
||||
class HeaderView : public QHeaderView {
|
||||
public:
|
||||
HeaderView(Qt::Orientation orientation, QWidget *parent = nullptr) : QHeaderView(orientation, parent) {}
|
||||
QSize sectionSizeFromContents(int logicalIndex) const override;
|
||||
void paintSection(QPainter *painter, const QRect &rect, int logicalIndex) const;
|
||||
};
|
||||
|
||||
class HistoryLogModel : public QAbstractTableModel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
HistoryLogModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
void setMessage(const MessageId &message_id);
|
||||
void updateState();
|
||||
void setFilter(int sig_idx, const QString &value, std::function<bool(double, double)> cmp);
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
void fetchMore(const QModelIndex &parent) override;
|
||||
inline bool canFetchMore(const QModelIndex &parent) const override { return has_more_data; }
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return messages.size(); }
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override {
|
||||
return display_signals_mode && !sigs.empty() ? sigs.size() + 1 : 2;
|
||||
}
|
||||
void refresh(bool fetch_message = true);
|
||||
|
||||
public slots:
|
||||
void setDisplayType(int type);
|
||||
void setDynamicMode(int state);
|
||||
void segmentsMerged();
|
||||
|
||||
public:
|
||||
struct Message {
|
||||
uint64_t mono_time = 0;
|
||||
QVector<double> sig_values;
|
||||
QByteArray data;
|
||||
QVector<QColor> colors;
|
||||
};
|
||||
|
||||
template <class InputIt>
|
||||
std::deque<HistoryLogModel::Message> fetchData(InputIt first, InputIt last, uint64_t min_time);
|
||||
std::deque<Message> fetchData(uint64_t from_time, uint64_t min_time = 0);
|
||||
|
||||
MessageId msg_id;
|
||||
CanData hex_colors;
|
||||
bool has_more_data = true;
|
||||
const int batch_size = 50;
|
||||
int filter_sig_idx = -1;
|
||||
double filter_value = 0;
|
||||
uint64_t last_fetch_time = 0;
|
||||
std::function<bool(double, double)> filter_cmp = nullptr;
|
||||
std::deque<Message> messages;
|
||||
std::vector<cabana::Signal *> sigs;
|
||||
bool dynamic_mode = true;
|
||||
bool display_signals_mode = true;
|
||||
};
|
||||
|
||||
class LogsWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LogsWidget(QWidget *parent);
|
||||
void setMessage(const MessageId &message_id);
|
||||
void updateState();
|
||||
void showEvent(QShowEvent *event) override;
|
||||
|
||||
private slots:
|
||||
void setFilter();
|
||||
|
||||
private:
|
||||
void refresh();
|
||||
|
||||
QTableView *logs;
|
||||
HistoryLogModel *model;
|
||||
QCheckBox *dynamic_mode;
|
||||
QComboBox *signals_cb, *comp_box, *display_type_cb;
|
||||
QLineEdit *value_edit;
|
||||
QWidget *filters_widget;
|
||||
MessageBytesDelegate *delegate;
|
||||
};
|
||||
114
tools/cabana/mainwin.h
Normal file
114
tools/cabana/mainwin.h
Normal file
@@ -0,0 +1,114 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDockWidget>
|
||||
#include <QJsonDocument>
|
||||
#include <QMainWindow>
|
||||
#include <QMenu>
|
||||
#include <QProgressBar>
|
||||
#include <QSplitter>
|
||||
#include <QStatusBar>
|
||||
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/detailwidget.h"
|
||||
#include "tools/cabana/messageswidget.h"
|
||||
#include "tools/cabana/videowidget.h"
|
||||
#include "tools/cabana/tools/findsimilarbits.h"
|
||||
|
||||
class MainWindow : public QMainWindow {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
MainWindow();
|
||||
void dockCharts(bool dock);
|
||||
void showStatusMessage(const QString &msg, int timeout = 0) { statusBar()->showMessage(msg, timeout); }
|
||||
void loadFile(const QString &fn, SourceSet s = SOURCE_ALL);
|
||||
ChartsWidget *charts_widget = nullptr;
|
||||
|
||||
public slots:
|
||||
void openStream();
|
||||
void closeStream();
|
||||
void changingStream();
|
||||
void streamStarted();
|
||||
|
||||
void newFile(SourceSet s = SOURCE_ALL);
|
||||
void openFile(SourceSet s = SOURCE_ALL);
|
||||
void openRecentFile();
|
||||
void loadDBCFromOpendbc(const QString &name);
|
||||
void save();
|
||||
void saveAs();
|
||||
void saveToClipboard();
|
||||
|
||||
signals:
|
||||
void showMessage(const QString &msg, int timeout);
|
||||
void updateProgressBar(uint64_t cur, uint64_t total, bool success);
|
||||
|
||||
protected:
|
||||
void remindSaveChanges();
|
||||
void closeFile(SourceSet s = SOURCE_ALL);
|
||||
void closeFile(DBCFile *dbc_file);
|
||||
void saveFile(DBCFile *dbc_file);
|
||||
void saveFileAs(DBCFile *dbc_file);
|
||||
void saveFileToClipboard(DBCFile *dbc_file);
|
||||
void loadFromClipboard(SourceSet s = SOURCE_ALL, bool close_all = true);
|
||||
void autoSave();
|
||||
void cleanupAutoSaveFile();
|
||||
void updateRecentFiles(const QString &fn);
|
||||
void updateRecentFileActions();
|
||||
void createActions();
|
||||
void createDockWindows();
|
||||
void createStatusBar();
|
||||
void createShortcuts();
|
||||
void closeEvent(QCloseEvent *event) override;
|
||||
void DBCFileChanged();
|
||||
void updateDownloadProgress(uint64_t cur, uint64_t total, bool success);
|
||||
void setOption();
|
||||
void findSimilarBits();
|
||||
void findSignal();
|
||||
void undoStackCleanChanged(bool clean);
|
||||
void undoStackIndexChanged(int index);
|
||||
void onlineHelp();
|
||||
void toggleFullScreen();
|
||||
void updateStatus();
|
||||
void updateLoadSaveMenus();
|
||||
void createDockWidgets();
|
||||
void eventsMerged();
|
||||
|
||||
VideoWidget *video_widget = nullptr;
|
||||
QDockWidget *video_dock;
|
||||
QDockWidget *messages_dock;
|
||||
MessagesWidget *messages_widget = nullptr;
|
||||
CenterWidget *center_widget;
|
||||
QWidget *floating_window = nullptr;
|
||||
QVBoxLayout *charts_layout;
|
||||
QProgressBar *progress_bar;
|
||||
QLabel *status_label;
|
||||
QJsonDocument fingerprint_to_dbc;
|
||||
QSplitter *video_splitter = nullptr;
|
||||
enum { MAX_RECENT_FILES = 15 };
|
||||
QAction *recent_files_acts[MAX_RECENT_FILES] = {};
|
||||
QMenu *open_recent_menu = nullptr;
|
||||
QMenu *manage_dbcs_menu = nullptr;
|
||||
QMenu *tools_menu = nullptr;
|
||||
QAction *close_stream_act = nullptr;
|
||||
QAction *save_dbc = nullptr;
|
||||
QAction *save_dbc_as = nullptr;
|
||||
QAction *copy_dbc_to_clipboard = nullptr;
|
||||
QString car_fingerprint;
|
||||
int prev_undostack_index = 0;
|
||||
int prev_undostack_count = 0;
|
||||
QByteArray default_state;
|
||||
friend class OnlineHelp;
|
||||
};
|
||||
|
||||
class HelpOverlay : public QWidget {
|
||||
Q_OBJECT
|
||||
public:
|
||||
HelpOverlay(MainWindow *parent);
|
||||
|
||||
protected:
|
||||
void drawHelpForWidget(QPainter &painter, QWidget *w);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
void mouseReleaseEvent(QMouseEvent *event) override;
|
||||
bool eventFilter(QObject *obj, QEvent *event) override;
|
||||
};
|
||||
119
tools/cabana/messageswidget.h
Normal file
119
tools/cabana/messageswidget.h
Normal file
@@ -0,0 +1,119 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <utility>
|
||||
#include <vector>
|
||||
|
||||
#include <QAbstractTableModel>
|
||||
#include <QCheckBox>
|
||||
#include <QContextMenuEvent>
|
||||
#include <QHeaderView>
|
||||
#include <QLabel>
|
||||
#include <QLineEdit>
|
||||
#include <QMenu>
|
||||
#include <QSet>
|
||||
#include <QTreeView>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class MessageListModel : public QAbstractTableModel {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
|
||||
enum Column {
|
||||
NAME = 0,
|
||||
SOURCE,
|
||||
ADDRESS,
|
||||
FREQ,
|
||||
COUNT,
|
||||
DATA,
|
||||
};
|
||||
|
||||
MessageListModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return Column::DATA + 1; }
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const;
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return msgs.size(); }
|
||||
void sort(int column, Qt::SortOrder order = Qt::AscendingOrder) override;
|
||||
void setFilterStrings(const QMap<int, QString> &filters);
|
||||
void msgsReceived(const QHash<MessageId, CanData> *new_msgs, bool has_new_ids);
|
||||
void fetchData();
|
||||
void suppress();
|
||||
void clearSuppress();
|
||||
void forceResetModel();
|
||||
void dbcModified();
|
||||
std::vector<MessageId> msgs;
|
||||
QSet<std::pair<MessageId, int>> suppressed_bytes;
|
||||
|
||||
private:
|
||||
void sortMessages(std::vector<MessageId> &new_msgs);
|
||||
bool matchMessage(const MessageId &id, const CanData &data, const QMap<int, QString> &filters);
|
||||
|
||||
QMap<int, QString> filter_str;
|
||||
QSet<uint32_t> dbc_address;
|
||||
int sort_column = 0;
|
||||
Qt::SortOrder sort_order = Qt::AscendingOrder;
|
||||
};
|
||||
|
||||
class MessageView : public QTreeView {
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageView(QWidget *parent) : QTreeView(parent) {}
|
||||
void drawRow(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void drawBranches(QPainter *painter, const QRect &rect, const QModelIndex &index) const override {}
|
||||
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles = QVector<int>()) override;
|
||||
void updateBytesSectionSize();
|
||||
void headerContextMenuEvent(const QPoint &pos);
|
||||
};
|
||||
|
||||
class MessageViewHeader : public QHeaderView {
|
||||
// https://stackoverflow.com/a/44346317
|
||||
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageViewHeader(QWidget *parent);
|
||||
void updateHeaderPositions();
|
||||
|
||||
void updateGeometries() override;
|
||||
QSize sizeHint() const override;
|
||||
|
||||
public slots:
|
||||
void clearFilters();
|
||||
|
||||
signals:
|
||||
void filtersUpdated(const QMap<int, QString> &filters);
|
||||
|
||||
private:
|
||||
void updateFilters();
|
||||
|
||||
QMap<int, QLineEdit *> editors;
|
||||
};
|
||||
|
||||
class MessagesWidget : public QWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
MessagesWidget(QWidget *parent);
|
||||
void selectMessage(const MessageId &message_id);
|
||||
QByteArray saveHeaderState() const { return view->header()->saveState(); }
|
||||
bool restoreHeaderState(const QByteArray &state) const { return view->header()->restoreState(state); }
|
||||
void updateSuppressedButtons();
|
||||
|
||||
public slots:
|
||||
void dbcModified();
|
||||
|
||||
signals:
|
||||
void msgSelectionChanged(const MessageId &message_id);
|
||||
|
||||
protected:
|
||||
MessageView *view;
|
||||
MessageViewHeader *header;
|
||||
std::optional<MessageId> current_msg_id;
|
||||
QCheckBox *multiple_lines_bytes;
|
||||
MessageListModel *model;
|
||||
QPushButton *suppress_add;
|
||||
QPushButton *suppress_clear;
|
||||
QLabel *num_msg_label;
|
||||
};
|
||||
70
tools/cabana/settings.h
Normal file
70
tools/cabana/settings.h
Normal file
@@ -0,0 +1,70 @@
|
||||
#pragma once
|
||||
|
||||
#include <QByteArray>
|
||||
#include <QCheckBox>
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QGroupBox>
|
||||
#include <QLineEdit>
|
||||
#include <QSpinBox>
|
||||
|
||||
#define LIGHT_THEME 1
|
||||
#define DARK_THEME 2
|
||||
|
||||
class Settings : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
enum DragDirection {
|
||||
MsbFirst,
|
||||
LsbFirst,
|
||||
AlwaysLE,
|
||||
AlwaysBE,
|
||||
};
|
||||
|
||||
Settings();
|
||||
void save();
|
||||
void load();
|
||||
|
||||
int fps = 10;
|
||||
int max_cached_minutes = 30;
|
||||
int chart_height = 200;
|
||||
int chart_column_count = 1;
|
||||
int chart_range = 3 * 60; // 3 minutes
|
||||
int chart_series_type = 0;
|
||||
int theme = 0;
|
||||
int sparkline_range = 15; // 15 seconds
|
||||
bool multiple_lines_bytes = true;
|
||||
bool log_livestream = true;
|
||||
bool suppress_defined_signals = false;
|
||||
QString log_path;
|
||||
QString last_dir;
|
||||
QString last_route_dir;
|
||||
QByteArray geometry;
|
||||
QByteArray video_splitter_state;
|
||||
QByteArray window_state;
|
||||
QStringList recent_files;
|
||||
QByteArray message_header_state;
|
||||
DragDirection drag_direction;
|
||||
|
||||
signals:
|
||||
void changed();
|
||||
};
|
||||
|
||||
class SettingsDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
SettingsDlg(QWidget *parent);
|
||||
void save();
|
||||
QSpinBox *fps;
|
||||
QSpinBox *cached_minutes;
|
||||
QSpinBox *chart_height;
|
||||
QComboBox *chart_series_type;
|
||||
QComboBox *theme;
|
||||
QGroupBox *log_livestream;
|
||||
QLineEdit *log_path;
|
||||
QComboBox *drag_direction;
|
||||
};
|
||||
|
||||
extern Settings settings;
|
||||
147
tools/cabana/signalview.h
Normal file
147
tools/cabana/signalview.h
Normal file
@@ -0,0 +1,147 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include <QAbstractItemModel>
|
||||
#include <QLabel>
|
||||
#include <QLineEdit>
|
||||
#include <QSlider>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QTableWidget>
|
||||
#include <QTreeView>
|
||||
|
||||
#include "tools/cabana/chart/chartswidget.h"
|
||||
#include "tools/cabana/chart/sparkline.h"
|
||||
|
||||
class SignalModel : public QAbstractItemModel {
|
||||
Q_OBJECT
|
||||
public:
|
||||
struct Item {
|
||||
enum Type {Root, Sig, Name, Size, Endian, Signed, Offset, Factor, SignalType, MultiplexValue, ExtraInfo, Unit, Comment, Min, Max, Desc };
|
||||
~Item() { qDeleteAll(children); }
|
||||
inline int row() { return parent->children.indexOf(this); }
|
||||
|
||||
Type type = Type::Root;
|
||||
Item *parent = nullptr;
|
||||
QList<Item *> children;
|
||||
|
||||
const cabana::Signal *sig = nullptr;
|
||||
QString title;
|
||||
bool highlight = false;
|
||||
bool extra_expanded = false;
|
||||
QString sig_val = "-";
|
||||
Sparkline sparkline;
|
||||
};
|
||||
|
||||
SignalModel(QObject *parent);
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return 2; }
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
QModelIndex index(int row, int column, const QModelIndex &parent = QModelIndex()) const override;
|
||||
QModelIndex parent(const QModelIndex &index) const override;
|
||||
Qt::ItemFlags flags(const QModelIndex &index) const override;
|
||||
bool setData(const QModelIndex &index, const QVariant &value, int role = Qt::EditRole) override;
|
||||
void setMessage(const MessageId &id);
|
||||
void setFilter(const QString &txt);
|
||||
bool saveSignal(const cabana::Signal *origin_s, cabana::Signal &s);
|
||||
Item *getItem(const QModelIndex &index) const;
|
||||
int signalRow(const cabana::Signal *sig) const;
|
||||
void showExtraInfo(const QModelIndex &index);
|
||||
|
||||
private:
|
||||
void insertItem(SignalModel::Item *parent_item, int pos, const cabana::Signal *sig);
|
||||
void handleSignalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void handleSignalUpdated(const cabana::Signal *sig);
|
||||
void handleSignalRemoved(const cabana::Signal *sig);
|
||||
void handleMsgChanged(MessageId id);
|
||||
void refresh();
|
||||
|
||||
MessageId msg_id;
|
||||
QString filter_str;
|
||||
std::unique_ptr<Item> root;
|
||||
friend class SignalView;
|
||||
friend class SignalItemDelegate;
|
||||
};
|
||||
|
||||
class ValueDescriptionDlg : public QDialog {
|
||||
public:
|
||||
ValueDescriptionDlg(const ValueDescription &descriptions, QWidget *parent);
|
||||
ValueDescription val_desc;
|
||||
|
||||
private:
|
||||
struct Delegate : public QStyledItemDelegate {
|
||||
Delegate(QWidget *parent) : QStyledItemDelegate(parent) {}
|
||||
QWidget *createEditor(QWidget *parent, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
};
|
||||
|
||||
void save();
|
||||
QTableWidget *table;
|
||||
};
|
||||
|
||||
class SignalItemDelegate : public QStyledItemDelegate {
|
||||
public:
|
||||
SignalItemDelegate(QObject *parent);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QSize sizeHint(const QStyleOptionViewItem &option, const QModelIndex &index) const;
|
||||
QWidget *createEditor(QWidget *parent, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void updateEditorGeometry(QWidget *editor, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
void setModelData(QWidget *editor, QAbstractItemModel *model, const QModelIndex &index) const override;
|
||||
|
||||
QValidator *name_validator, *double_validator;
|
||||
QFont label_font, minmax_font;
|
||||
const int color_label_width = 18;
|
||||
mutable QSize button_size;
|
||||
mutable QHash<QString, int> width_cache;
|
||||
};
|
||||
|
||||
class SignalView : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
SignalView(ChartsWidget *charts, QWidget *parent);
|
||||
void setMessage(const MessageId &id);
|
||||
void signalHovered(const cabana::Signal *sig);
|
||||
void updateChartState();
|
||||
void selectSignal(const cabana::Signal *sig, bool expand = false);
|
||||
void rowClicked(const QModelIndex &index);
|
||||
SignalModel *model = nullptr;
|
||||
|
||||
signals:
|
||||
void highlight(const cabana::Signal *sig);
|
||||
void showChart(const MessageId &id, const cabana::Signal *sig, bool show, bool merge);
|
||||
|
||||
private:
|
||||
void rowsChanged();
|
||||
void resizeEvent(QResizeEvent* event) override;
|
||||
void updateToolBar();
|
||||
void setSparklineRange(int value);
|
||||
void handleSignalAdded(MessageId id, const cabana::Signal *sig);
|
||||
void handleSignalUpdated(const cabana::Signal *sig);
|
||||
void updateState(const QHash<MessageId, CanData> *msgs = nullptr);
|
||||
|
||||
struct TreeView : public QTreeView {
|
||||
TreeView(QWidget *parent) : QTreeView(parent) {}
|
||||
void rowsInserted(const QModelIndex &parent, int start, int end) override {
|
||||
((SignalView *)parentWidget())->rowsChanged();
|
||||
// update widget geometries in QTreeView::rowsInserted
|
||||
QTreeView::rowsInserted(parent, start, end);
|
||||
}
|
||||
void dataChanged(const QModelIndex &topLeft, const QModelIndex &bottomRight, const QVector<int> &roles = QVector<int>()) override {
|
||||
// Bypass the slow call to QTreeView::dataChanged.
|
||||
QAbstractItemView::dataChanged(topLeft, bottomRight, roles);
|
||||
}
|
||||
void leaveEvent(QEvent *event) override {
|
||||
emit static_cast<SignalView *>(parentWidget())->highlight(nullptr);
|
||||
QTreeView::leaveEvent(event);
|
||||
}
|
||||
};
|
||||
int max_value_width = 0;
|
||||
TreeView *tree;
|
||||
QLabel *sparkline_label;
|
||||
QSlider *sparkline_range_slider;
|
||||
QLineEdit *filter_edit;
|
||||
ChartsWidget *charts;
|
||||
QLabel *signal_count_lb;
|
||||
SignalItemDelegate *delegate;
|
||||
friend SignalItemDelegate;
|
||||
};
|
||||
136
tools/cabana/streams/abstractstream.h
Normal file
136
tools/cabana/streams/abstractstream.h
Normal file
@@ -0,0 +1,136 @@
|
||||
#pragma once
|
||||
|
||||
#include <array>
|
||||
#include <atomic>
|
||||
#include <deque>
|
||||
#include <memory>
|
||||
#include <tuple>
|
||||
#include <unordered_map>
|
||||
#include <vector>
|
||||
|
||||
#include <QColor>
|
||||
#include <QHash>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
#include "tools/cabana/util.h"
|
||||
#include "tools/replay/replay.h"
|
||||
|
||||
struct CanData {
|
||||
void compute(const char *dat, const int size, double current_sec, double playback_speed, const std::vector<uint8_t> *mask, uint32_t in_freq = 0);
|
||||
|
||||
double ts = 0.;
|
||||
uint32_t count = 0;
|
||||
double freq = 0;
|
||||
QByteArray dat;
|
||||
QVector<QColor> colors;
|
||||
std::vector<double> last_change_t;
|
||||
std::vector<std::array<uint32_t, 8>> bit_change_counts;
|
||||
std::vector<int> last_delta;
|
||||
std::vector<int> same_delta_counter;
|
||||
};
|
||||
|
||||
struct CanEvent {
|
||||
uint8_t src;
|
||||
uint32_t address;
|
||||
uint64_t mono_time;
|
||||
uint8_t size;
|
||||
uint8_t dat[];
|
||||
};
|
||||
|
||||
struct BusConfig {
|
||||
int can_speed_kbps = 500;
|
||||
int data_speed_kbps = 2000;
|
||||
bool can_fd = false;
|
||||
};
|
||||
|
||||
class AbstractStream : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
AbstractStream(QObject *parent);
|
||||
virtual ~AbstractStream() {}
|
||||
virtual void start() = 0;
|
||||
inline bool liveStreaming() const { return route() == nullptr; }
|
||||
virtual void seekTo(double ts) {}
|
||||
virtual QString routeName() const = 0;
|
||||
virtual QString carFingerprint() const { return ""; }
|
||||
virtual double routeStartTime() const { return 0; }
|
||||
virtual double currentSec() const = 0;
|
||||
virtual double totalSeconds() const { return lastEventMonoTime() / 1e9 - routeStartTime(); }
|
||||
const CanData &lastMessage(const MessageId &id);
|
||||
virtual VisionStreamType visionStreamType() const { return VISION_STREAM_ROAD; }
|
||||
virtual const Route *route() const { return nullptr; }
|
||||
virtual void setSpeed(float speed) {}
|
||||
virtual double getSpeed() { return 1; }
|
||||
virtual bool isPaused() const { return false; }
|
||||
virtual void pause(bool pause) {}
|
||||
const std::vector<const CanEvent *> &allEvents() const { return all_events_; }
|
||||
const std::vector<const CanEvent *> &events(const MessageId &id) const;
|
||||
virtual const std::vector<std::tuple<double, double, TimelineType>> getTimeline() { return {}; }
|
||||
|
||||
signals:
|
||||
void paused();
|
||||
void resume();
|
||||
void seekedTo(double sec);
|
||||
void streamStarted();
|
||||
void eventsMerged();
|
||||
void updated();
|
||||
void msgsReceived(const QHash<MessageId, CanData> *new_msgs, bool has_new_ids);
|
||||
void sourcesUpdated(const SourceSet &s);
|
||||
|
||||
public:
|
||||
QHash<MessageId, CanData> last_msgs;
|
||||
SourceSet sources;
|
||||
|
||||
protected:
|
||||
void mergeEvents(std::vector<Event *>::const_iterator first, std::vector<Event *>::const_iterator last);
|
||||
bool postEvents();
|
||||
uint64_t lastEventMonoTime() const { return lastest_event_ts; }
|
||||
void updateEvent(const MessageId &id, double sec, const uint8_t *data, uint8_t size);
|
||||
void updateMessages(QHash<MessageId, CanData> *);
|
||||
void updateMasks();
|
||||
void updateLastMsgsTo(double sec);
|
||||
|
||||
uint64_t lastest_event_ts = 0;
|
||||
std::atomic<bool> processing = false;
|
||||
std::unique_ptr<QHash<MessageId, CanData>> new_msgs;
|
||||
QHash<MessageId, CanData> all_msgs;
|
||||
std::unordered_map<MessageId, std::vector<const CanEvent *>> events_;
|
||||
std::vector<const CanEvent *> all_events_;
|
||||
std::deque<std::unique_ptr<char[]>> memory_blocks;
|
||||
std::mutex mutex;
|
||||
std::unordered_map<MessageId, std::vector<uint8_t>> masks;
|
||||
};
|
||||
|
||||
class AbstractOpenStreamWidget : public QWidget {
|
||||
public:
|
||||
AbstractOpenStreamWidget(AbstractStream **stream, QWidget *parent = nullptr) : stream(stream), QWidget(parent) {}
|
||||
virtual bool open() = 0;
|
||||
virtual QString title() = 0;
|
||||
|
||||
protected:
|
||||
AbstractStream **stream = nullptr;
|
||||
};
|
||||
|
||||
class DummyStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DummyStream(QObject *parent) : AbstractStream(parent) {}
|
||||
QString routeName() const override { return tr("No Stream"); }
|
||||
void start() override { emit streamStarted(); }
|
||||
double currentSec() const override { return 0; }
|
||||
};
|
||||
|
||||
class StreamNotifier : public QObject {
|
||||
Q_OBJECT
|
||||
public:
|
||||
StreamNotifier(QObject *parent = nullptr) : QObject(parent) {}
|
||||
static StreamNotifier* instance();
|
||||
signals:
|
||||
void streamStarted();
|
||||
void changingStream();
|
||||
};
|
||||
|
||||
// A global pointer referring to the unique AbstractStream object
|
||||
extern AbstractStream *can;
|
||||
30
tools/cabana/streams/devicestream.h
Normal file
30
tools/cabana/streams/devicestream.h
Normal file
@@ -0,0 +1,30 @@
|
||||
#pragma once
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
|
||||
class DeviceStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DeviceStream(QObject *parent, QString address = {});
|
||||
static AbstractOpenStreamWidget *widget(AbstractStream **stream);
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From %1").arg(zmq_address.isEmpty() ? "127.0.0.1" : zmq_address);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
const QString zmq_address;
|
||||
};
|
||||
|
||||
class OpenDeviceWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenDeviceWidget(AbstractStream **stream);
|
||||
bool open() override;
|
||||
QString title() override { return tr("&Device"); }
|
||||
|
||||
private:
|
||||
QLineEdit *ip_address;
|
||||
QButtonGroup *group;
|
||||
};
|
||||
62
tools/cabana/streams/livestream.h
Normal file
62
tools/cabana/streams/livestream.h
Normal file
@@ -0,0 +1,62 @@
|
||||
#pragma once
|
||||
|
||||
#include <deque>
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include <QBasicTimer>
|
||||
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class LiveStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LiveStream(QObject *parent);
|
||||
virtual ~LiveStream();
|
||||
void start() override;
|
||||
inline double routeStartTime() const override { return begin_event_ts / 1e9; }
|
||||
inline double currentSec() const override { return (current_event_ts - begin_event_ts) / 1e9; }
|
||||
void setSpeed(float speed) override { speed_ = speed; }
|
||||
double getSpeed() override { return speed_; }
|
||||
bool isPaused() const override { return paused_; }
|
||||
void pause(bool pause) override;
|
||||
void seekTo(double sec) override;
|
||||
|
||||
protected:
|
||||
virtual void streamThread() = 0;
|
||||
void handleEvent(const char *data, const size_t size);
|
||||
|
||||
private:
|
||||
void startUpdateTimer();
|
||||
void timerEvent(QTimerEvent *event) override;
|
||||
void updateEvents();
|
||||
|
||||
struct Msg {
|
||||
Msg(const char *data, const size_t size) {
|
||||
event = ::new Event(aligned_buf.align(data, size));
|
||||
}
|
||||
~Msg() { ::delete event; }
|
||||
Event *event;
|
||||
AlignedBuffer aligned_buf;
|
||||
};
|
||||
|
||||
std::mutex lock;
|
||||
QThread *stream_thread;
|
||||
std::vector<Event *> receivedEvents;
|
||||
std::deque<Msg> receivedMessages;
|
||||
|
||||
int timer_id;
|
||||
QBasicTimer update_timer;
|
||||
|
||||
uint64_t begin_event_ts = 0;
|
||||
uint64_t current_event_ts = 0;
|
||||
uint64_t first_event_ts = 0;
|
||||
uint64_t first_update_ts = 0;
|
||||
bool post_last_event = true;
|
||||
double speed_ = 1;
|
||||
bool paused_ = false;
|
||||
|
||||
struct Logger;
|
||||
std::unique_ptr<Logger> logger;
|
||||
};
|
||||
53
tools/cabana/streams/pandastream.h
Normal file
53
tools/cabana/streams/pandastream.h
Normal file
@@ -0,0 +1,53 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
#include <vector>
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QFormLayout>
|
||||
#include <QVBoxLayout>
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
#include "selfdrive/boardd/panda.h"
|
||||
|
||||
const uint32_t speeds[] = {10U, 20U, 50U, 100U, 125U, 250U, 500U, 1000U};
|
||||
const uint32_t data_speeds[] = {10U, 20U, 50U, 100U, 125U, 250U, 500U, 1000U, 2000U, 5000U};
|
||||
|
||||
struct PandaStreamConfig {
|
||||
QString serial = "";
|
||||
std::vector<BusConfig> bus_config;
|
||||
};
|
||||
|
||||
class PandaStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
PandaStream(QObject *parent, PandaStreamConfig config_ = {});
|
||||
static AbstractOpenStreamWidget *widget(AbstractStream **stream);
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From Panda %1").arg(config.serial);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
bool connect();
|
||||
|
||||
std::unique_ptr<Panda> panda;
|
||||
PandaStreamConfig config = {};
|
||||
};
|
||||
|
||||
class OpenPandaWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenPandaWidget(AbstractStream **stream);
|
||||
bool open() override;
|
||||
QString title() override { return tr("&Panda"); }
|
||||
|
||||
private:
|
||||
void refreshSerials();
|
||||
void buildConfigForm();
|
||||
|
||||
QComboBox *serial_edit;
|
||||
QFormLayout *config_layout;
|
||||
PandaStreamConfig config = {};
|
||||
};
|
||||
53
tools/cabana/streams/replaystream.h
Normal file
53
tools/cabana/streams/replaystream.h
Normal file
@@ -0,0 +1,53 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <memory>
|
||||
#include <set>
|
||||
#include <tuple>
|
||||
#include <vector>
|
||||
|
||||
#include "common/prefix.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class ReplayStream : public AbstractStream {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
ReplayStream(QObject *parent);
|
||||
void start() override;
|
||||
bool loadRoute(const QString &route, const QString &data_dir, uint32_t replay_flags = REPLAY_FLAG_NONE);
|
||||
bool eventFilter(const Event *event);
|
||||
void seekTo(double ts) override { replay->seekTo(std::max(double(0), ts), false); }
|
||||
inline QString routeName() const override { return replay->route()->name(); }
|
||||
inline QString carFingerprint() const override { return replay->carFingerprint().c_str(); }
|
||||
double totalSeconds() const override { return replay->totalSeconds(); }
|
||||
inline VisionStreamType visionStreamType() const override { return replay->hasFlag(REPLAY_FLAG_ECAM) ? VISION_STREAM_WIDE_ROAD : VISION_STREAM_ROAD; }
|
||||
inline double routeStartTime() const override { return replay->routeStartTime() / (double)1e9; }
|
||||
inline double currentSec() const override { return replay->currentSeconds(); }
|
||||
inline const Route *route() const override { return replay->route(); }
|
||||
inline void setSpeed(float speed) override { replay->setSpeed(speed); }
|
||||
inline float getSpeed() const { return replay->getSpeed(); }
|
||||
inline bool isPaused() const override { return replay->isPaused(); }
|
||||
void pause(bool pause) override;
|
||||
inline const std::vector<std::tuple<double, double, TimelineType>> getTimeline() override { return replay->getTimeline(); }
|
||||
static AbstractOpenStreamWidget *widget(AbstractStream **stream);
|
||||
|
||||
private:
|
||||
void mergeSegments();
|
||||
std::unique_ptr<Replay> replay = nullptr;
|
||||
std::set<int> processed_segments;
|
||||
std::unique_ptr<OpenpilotPrefix> op_prefix;
|
||||
};
|
||||
|
||||
class OpenReplayWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenReplayWidget(AbstractStream **stream);
|
||||
bool open() override;
|
||||
QString title() override { return tr("&Replay"); }
|
||||
|
||||
private:
|
||||
QLineEdit *route_edit;
|
||||
QComboBox *choose_video_cb;
|
||||
};
|
||||
52
tools/cabana/streams/socketcanstream.h
Normal file
52
tools/cabana/streams/socketcanstream.h
Normal file
@@ -0,0 +1,52 @@
|
||||
#pragma once
|
||||
|
||||
#include <memory>
|
||||
|
||||
#include <QtSerialBus/QCanBus>
|
||||
#include <QtSerialBus/QCanBusDevice>
|
||||
#include <QtSerialBus/QCanBusDeviceInfo>
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QFormLayout>
|
||||
#include <QVBoxLayout>
|
||||
|
||||
#include "tools/cabana/streams/livestream.h"
|
||||
|
||||
struct SocketCanStreamConfig {
|
||||
QString device = ""; // TODO: support multiple devices/buses at once
|
||||
};
|
||||
|
||||
class SocketCanStream : public LiveStream {
|
||||
Q_OBJECT
|
||||
public:
|
||||
SocketCanStream(QObject *parent, SocketCanStreamConfig config_ = {});
|
||||
static AbstractOpenStreamWidget *widget(AbstractStream **stream);
|
||||
|
||||
static bool available();
|
||||
|
||||
inline QString routeName() const override {
|
||||
return QString("Live Streaming From Socket CAN %1").arg(config.device);
|
||||
}
|
||||
|
||||
protected:
|
||||
void streamThread() override;
|
||||
bool connect();
|
||||
|
||||
SocketCanStreamConfig config = {};
|
||||
std::unique_ptr<QCanBusDevice> device;
|
||||
};
|
||||
|
||||
class OpenSocketCanWidget : public AbstractOpenStreamWidget {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
OpenSocketCanWidget(AbstractStream **stream);
|
||||
bool open() override;
|
||||
QString title() override { return tr("&SocketCAN"); }
|
||||
|
||||
private:
|
||||
void refreshDevices();
|
||||
|
||||
QComboBox *device_edit;
|
||||
SocketCanStreamConfig config = {};
|
||||
};
|
||||
20
tools/cabana/streamselector.h
Normal file
20
tools/cabana/streamselector.h
Normal file
@@ -0,0 +1,20 @@
|
||||
#pragma once
|
||||
|
||||
#include <QDialog>
|
||||
#include <QLineEdit>
|
||||
#include <QTabWidget>
|
||||
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
class StreamSelector : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
StreamSelector(AbstractStream **stream, QWidget *parent = nullptr);
|
||||
void addStreamWidget(AbstractOpenStreamWidget *w);
|
||||
QString dbcFile() const { return dbc_file->text(); }
|
||||
|
||||
private:
|
||||
QLineEdit *dbc_file;
|
||||
QTabWidget *tab;
|
||||
};
|
||||
63
tools/cabana/tools/findsignal.h
Normal file
63
tools/cabana/tools/findsignal.h
Normal file
@@ -0,0 +1,63 @@
|
||||
#pragma once
|
||||
|
||||
#include <algorithm>
|
||||
#include <limits>
|
||||
|
||||
#include <QAbstractTableModel>
|
||||
#include <QLabel>
|
||||
#include <QPushButton>
|
||||
#include <QTableView>
|
||||
|
||||
#include "tools/cabana/commands.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
|
||||
class FindSignalModel : public QAbstractTableModel {
|
||||
public:
|
||||
struct SearchSignal {
|
||||
MessageId id = {};
|
||||
uint64_t mono_time = 0;
|
||||
cabana::Signal sig = {};
|
||||
double value = 0.;
|
||||
QStringList values;
|
||||
};
|
||||
|
||||
FindSignalModel(QObject *parent) : QAbstractTableModel(parent) {}
|
||||
QVariant headerData(int section, Qt::Orientation orientation, int role = Qt::DisplayRole) const override;
|
||||
QVariant data(const QModelIndex &index, int role = Qt::DisplayRole) const override;
|
||||
int columnCount(const QModelIndex &parent = QModelIndex()) const override { return 3; }
|
||||
int rowCount(const QModelIndex &parent = QModelIndex()) const override { return std::min(filtered_signals.size(), 300); }
|
||||
void search(std::function<bool(double)> cmp);
|
||||
void reset();
|
||||
void undo();
|
||||
|
||||
QList<SearchSignal> filtered_signals;
|
||||
QList<SearchSignal> initial_signals;
|
||||
QList<QList<SearchSignal>> histories;
|
||||
uint64_t last_time = std::numeric_limits<uint64_t>::max();
|
||||
};
|
||||
|
||||
class FindSignalDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
public:
|
||||
FindSignalDlg(QWidget *parent);
|
||||
|
||||
signals:
|
||||
void openMessage(const MessageId &id);
|
||||
|
||||
private:
|
||||
void search();
|
||||
void modelReset();
|
||||
void setInitialSignals();
|
||||
void customMenuRequested(const QPoint &pos);
|
||||
|
||||
QLineEdit *value1, *value2, *factor_edit, *offset_edit;
|
||||
QLineEdit *bus_edit, *address_edit, *first_time_edit, *last_time_edit;
|
||||
QComboBox *compare_cb;
|
||||
QSpinBox *min_size, *max_size;
|
||||
QCheckBox *litter_endian, *is_signed;
|
||||
QPushButton *search_btn, *reset_btn, *undo_btn;
|
||||
QGroupBox *properties_group, *message_group;
|
||||
QTableView *view;
|
||||
QLabel *to_label, *stats_label;
|
||||
FindSignalModel *model;
|
||||
};
|
||||
34
tools/cabana/tools/findsimilarbits.h
Normal file
34
tools/cabana/tools/findsimilarbits.h
Normal file
@@ -0,0 +1,34 @@
|
||||
#pragma once
|
||||
|
||||
#include <QComboBox>
|
||||
#include <QDialog>
|
||||
#include <QLineEdit>
|
||||
#include <QSpinBox>
|
||||
#include <QTableWidget>
|
||||
|
||||
#include "tools/cabana/dbc/dbcmanager.h"
|
||||
|
||||
class FindSimilarBitsDlg : public QDialog {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
FindSimilarBitsDlg(QWidget *parent);
|
||||
|
||||
signals:
|
||||
void openMessage(const MessageId &msg_id);
|
||||
|
||||
private:
|
||||
struct mismatched_struct {
|
||||
uint32_t address, byte_idx, bit_idx, mismatches, total;
|
||||
float perc;
|
||||
};
|
||||
QList<mismatched_struct> calcBits(uint8_t bus, uint32_t selected_address, int byte_idx, int bit_idx, uint8_t find_bus,
|
||||
bool equal, int min_msgs_cnt);
|
||||
void find();
|
||||
|
||||
QTableWidget *table;
|
||||
QComboBox *src_bus_combo, *find_bus_combo, *msg_cb, *equal_combo;
|
||||
QSpinBox *byte_idx_sb, *bit_idx_sb;
|
||||
QPushButton *search_btn;
|
||||
QLineEdit *min_msgs;
|
||||
};
|
||||
157
tools/cabana/util.h
Normal file
157
tools/cabana/util.h
Normal file
@@ -0,0 +1,157 @@
|
||||
#pragma once
|
||||
|
||||
#include <cmath>
|
||||
#include <vector>
|
||||
#include <utility>
|
||||
|
||||
#include <QApplication>
|
||||
#include <QByteArray>
|
||||
#include <QDateTime>
|
||||
#include <QDoubleValidator>
|
||||
#include <QFont>
|
||||
#include <QRegExpValidator>
|
||||
#include <QSocketNotifier>
|
||||
#include <QStringBuilder>
|
||||
#include <QStyledItemDelegate>
|
||||
#include <QToolButton>
|
||||
#include <QVector>
|
||||
|
||||
#include "tools/cabana/dbc/dbc.h"
|
||||
#include "tools/cabana/settings.h"
|
||||
|
||||
class LogSlider : public QSlider {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
LogSlider(double factor, Qt::Orientation orientation, QWidget *parent = nullptr) : factor(factor), QSlider(orientation, parent) {}
|
||||
|
||||
void setRange(double min, double max) {
|
||||
log_min = factor * std::log10(min);
|
||||
log_max = factor * std::log10(max);
|
||||
QSlider::setRange(min, max);
|
||||
setValue(QSlider::value());
|
||||
}
|
||||
int value() const {
|
||||
double v = log_min + (log_max - log_min) * ((QSlider::value() - minimum()) / double(maximum() - minimum()));
|
||||
return std::lround(std::pow(10, v / factor));
|
||||
}
|
||||
void setValue(int v) {
|
||||
double log_v = std::clamp(factor * std::log10(v), log_min, log_max);
|
||||
v = minimum() + (maximum() - minimum()) * ((log_v - log_min) / (log_max - log_min));
|
||||
QSlider::setValue(v);
|
||||
}
|
||||
|
||||
private:
|
||||
double factor, log_min = 0, log_max = 1;
|
||||
};
|
||||
|
||||
enum {
|
||||
ColorsRole = Qt::UserRole + 1,
|
||||
BytesRole = Qt::UserRole + 2
|
||||
};
|
||||
|
||||
class SegmentTree {
|
||||
public:
|
||||
SegmentTree() = default;
|
||||
void build(const QVector<QPointF> &arr);
|
||||
inline std::pair<double, double> minmax(int left, int right) const { return get_minmax(1, 0, size - 1, left, right); }
|
||||
|
||||
private:
|
||||
std::pair<double, double> get_minmax(int n, int left, int right, int range_left, int range_right) const;
|
||||
void build_tree(const QVector<QPointF> &arr, int n, int left, int right);
|
||||
std::vector<std::pair<double ,double>> tree;
|
||||
int size = 0;
|
||||
};
|
||||
|
||||
class MessageBytesDelegate : public QStyledItemDelegate {
|
||||
Q_OBJECT
|
||||
public:
|
||||
MessageBytesDelegate(QObject *parent, bool multiple_lines = false);
|
||||
void paint(QPainter *painter, const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
QSize sizeHint(const QStyleOptionViewItem &option, const QModelIndex &index) const override;
|
||||
bool multipleLines() const { return multiple_lines; }
|
||||
void setMultipleLines(bool v) { multiple_lines = v; }
|
||||
int widthForBytes(int n) const;
|
||||
|
||||
private:
|
||||
QFont fixed_font;
|
||||
QSize byte_size = {};
|
||||
bool multiple_lines = false;
|
||||
};
|
||||
|
||||
inline QString toHex(const QByteArray &dat) { return dat.toHex(' ').toUpper(); }
|
||||
QString toHex(uint8_t byte);
|
||||
|
||||
class NameValidator : public QRegExpValidator {
|
||||
Q_OBJECT
|
||||
public:
|
||||
NameValidator(QObject *parent=nullptr);
|
||||
QValidator::State validate(QString &input, int &pos) const override;
|
||||
};
|
||||
|
||||
class DoubleValidator : public QDoubleValidator {
|
||||
Q_OBJECT
|
||||
public:
|
||||
DoubleValidator(QObject *parent = nullptr);
|
||||
};
|
||||
|
||||
namespace utils {
|
||||
QPixmap icon(const QString &id);
|
||||
void setTheme(int theme);
|
||||
inline QString formatSeconds(int seconds) {
|
||||
return QDateTime::fromSecsSinceEpoch(seconds, Qt::UTC).toString(seconds > 60 * 60 ? "hh:mm:ss" : "mm:ss");
|
||||
}
|
||||
}
|
||||
|
||||
class ToolButton : public QToolButton {
|
||||
Q_OBJECT
|
||||
public:
|
||||
ToolButton(const QString &icon, const QString &tooltip = {}, QWidget *parent = nullptr) : QToolButton(parent) {
|
||||
setIcon(icon);
|
||||
setToolTip(tooltip);
|
||||
setAutoRaise(true);
|
||||
const int metric = QApplication::style()->pixelMetric(QStyle::PM_SmallIconSize);
|
||||
setIconSize({metric, metric});
|
||||
theme = settings.theme;
|
||||
connect(&settings, &Settings::changed, this, &ToolButton::updateIcon);
|
||||
}
|
||||
void setIcon(const QString &icon) {
|
||||
icon_str = icon;
|
||||
QToolButton::setIcon(utils::icon(icon_str));
|
||||
}
|
||||
|
||||
private:
|
||||
void updateIcon() { if (std::exchange(theme, settings.theme) != theme) setIcon(icon_str); }
|
||||
QString icon_str;
|
||||
int theme;
|
||||
};
|
||||
|
||||
class TabBar : public QTabBar {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
TabBar(QWidget *parent) : QTabBar(parent) {}
|
||||
int addTab(const QString &text);
|
||||
|
||||
private:
|
||||
void closeTabClicked();
|
||||
};
|
||||
|
||||
class UnixSignalHandler : public QObject {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
UnixSignalHandler(QObject *parent = nullptr);
|
||||
~UnixSignalHandler();
|
||||
static void signalHandler(int s);
|
||||
|
||||
public slots:
|
||||
void handleSigTerm();
|
||||
|
||||
private:
|
||||
inline static int sig_fd[2] = {};
|
||||
QSocketNotifier *sn;
|
||||
};
|
||||
|
||||
int num_decimals(double num);
|
||||
QString signalToolTip(const cabana::Signal *sig);
|
||||
87
tools/cabana/videowidget.h
Normal file
87
tools/cabana/videowidget.h
Normal file
@@ -0,0 +1,87 @@
|
||||
#pragma once
|
||||
|
||||
#include <atomic>
|
||||
#include <map>
|
||||
#include <memory>
|
||||
#include <mutex>
|
||||
#include <tuple>
|
||||
#include <vector>
|
||||
|
||||
#include <QFuture>
|
||||
#include <QLabel>
|
||||
#include <QPushButton>
|
||||
#include <QSlider>
|
||||
|
||||
#include "selfdrive/ui/qt/widgets/cameraview.h"
|
||||
#include "tools/cabana/streams/abstractstream.h"
|
||||
|
||||
struct AlertInfo {
|
||||
cereal::ControlsState::AlertStatus status;
|
||||
QString text1;
|
||||
QString text2;
|
||||
};
|
||||
|
||||
class InfoLabel : public QWidget {
|
||||
public:
|
||||
InfoLabel(QWidget *parent);
|
||||
void showPixmap(const QPoint &pt, const QString &sec, const QPixmap &pm, const AlertInfo &alert);
|
||||
void showAlert(const AlertInfo &alert);
|
||||
void paintEvent(QPaintEvent *event) override;
|
||||
QPixmap pixmap;
|
||||
QString second;
|
||||
AlertInfo alert_info;
|
||||
};
|
||||
|
||||
class Slider : public QSlider {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
Slider(QWidget *parent);
|
||||
~Slider();
|
||||
double currentSecond() const { return value() / factor; }
|
||||
void setCurrentSecond(double sec) { setValue(sec * factor); }
|
||||
void setTimeRange(double min, double max);
|
||||
AlertInfo alertInfo(double sec);
|
||||
QPixmap thumbnail(double sec);
|
||||
|
||||
signals:
|
||||
void updateMaximumTime(double);
|
||||
|
||||
private:
|
||||
void mousePressEvent(QMouseEvent *e) override;
|
||||
void mouseMoveEvent(QMouseEvent *e) override;
|
||||
bool event(QEvent *event) override;
|
||||
void paintEvent(QPaintEvent *ev) override;
|
||||
void parseQLog();
|
||||
|
||||
const double factor = 1000.0;
|
||||
std::vector<std::tuple<double, double, TimelineType>> timeline;
|
||||
std::mutex thumbnail_lock;
|
||||
std::atomic<bool> abort_parse_qlog = false;
|
||||
QMap<uint64_t, QPixmap> thumbnails;
|
||||
std::map<uint64_t, AlertInfo> alerts;
|
||||
std::unique_ptr<QFuture<void>> qlog_future;
|
||||
InfoLabel thumbnail_label;
|
||||
};
|
||||
|
||||
class VideoWidget : public QFrame {
|
||||
Q_OBJECT
|
||||
|
||||
public:
|
||||
VideoWidget(QWidget *parnet = nullptr);
|
||||
void updateTimeRange(double min, double max, bool is_zommed);
|
||||
void setMaximumTime(double sec);
|
||||
|
||||
protected:
|
||||
void updateState();
|
||||
void updatePlayBtnState();
|
||||
QWidget *createCameraWidget();
|
||||
|
||||
CameraWidget *cam_widget;
|
||||
double maximum_time = 0;
|
||||
QLabel *end_time_label;
|
||||
QLabel *time_label;
|
||||
QPushButton *play_btn;
|
||||
InfoLabel *alert_label;
|
||||
Slider *slider;
|
||||
};
|
||||
140
tools/camerastream/compressed_vipc.py
Executable file
140
tools/camerastream/compressed_vipc.py
Executable file
@@ -0,0 +1,140 @@
|
||||
#!/usr/bin/env python3
|
||||
import av
|
||||
import os
|
||||
import sys
|
||||
import argparse
|
||||
import numpy as np
|
||||
import multiprocessing
|
||||
import time
|
||||
|
||||
import cereal.messaging as messaging
|
||||
from cereal.visionipc import VisionIpcServer, VisionStreamType
|
||||
|
||||
W, H = 1928, 1208
|
||||
V4L2_BUF_FLAG_KEYFRAME = 8
|
||||
|
||||
ENCODE_SOCKETS = {
|
||||
VisionStreamType.VISION_STREAM_ROAD: "roadEncodeData",
|
||||
VisionStreamType.VISION_STREAM_WIDE_ROAD: "wideRoadEncodeData",
|
||||
VisionStreamType.VISION_STREAM_DRIVER: "driverEncodeData",
|
||||
}
|
||||
|
||||
def decoder(addr, vipc_server, vst, nvidia, debug=False):
|
||||
sock_name = ENCODE_SOCKETS[vst]
|
||||
if debug:
|
||||
print("start decoder for %s" % sock_name)
|
||||
if nvidia:
|
||||
os.environ["NV_LOW_LATENCY"] = "3" # both bLowLatency and CUVID_PKT_ENDOFPICTURE
|
||||
sys.path += os.environ["LD_LIBRARY_PATH"].split(":")
|
||||
import PyNvCodec as nvc # pylint: disable=import-error
|
||||
|
||||
nvDec = nvc.PyNvDecoder(W, H, nvc.PixelFormat.NV12, nvc.CudaVideoCodec.HEVC, 0)
|
||||
cc1 = nvc.ColorspaceConversionContext(nvc.ColorSpace.BT_709, nvc.ColorRange.JPEG)
|
||||
conv_yuv = nvc.PySurfaceConverter(W, H, nvc.PixelFormat.NV12, nvc.PixelFormat.YUV420, 0)
|
||||
nvDwn_yuv = nvc.PySurfaceDownloader(W, H, nvc.PixelFormat.YUV420, 0)
|
||||
img_yuv = np.ndarray((H*W//2*3), dtype=np.uint8)
|
||||
else:
|
||||
codec = av.CodecContext.create("hevc", "r")
|
||||
|
||||
os.environ["ZMQ"] = "1"
|
||||
messaging.context = messaging.Context()
|
||||
sock = messaging.sub_sock(sock_name, None, addr=addr, conflate=False)
|
||||
cnt = 0
|
||||
last_idx = -1
|
||||
seen_iframe = False
|
||||
|
||||
time_q = []
|
||||
while 1:
|
||||
msgs = messaging.drain_sock(sock, wait_for_one=True)
|
||||
for evt in msgs:
|
||||
evta = getattr(evt, evt.which())
|
||||
if debug and evta.idx.encodeId != 0 and evta.idx.encodeId != (last_idx+1):
|
||||
print("DROP PACKET!")
|
||||
last_idx = evta.idx.encodeId
|
||||
if not seen_iframe and not (evta.idx.flags & V4L2_BUF_FLAG_KEYFRAME):
|
||||
if debug:
|
||||
print("waiting for iframe")
|
||||
continue
|
||||
time_q.append(time.monotonic())
|
||||
network_latency = (int(time.time()*1e9) - evta.unixTimestampNanos)/1e6
|
||||
frame_latency = ((evta.idx.timestampEof/1e9) - (evta.idx.timestampSof/1e9))*1000
|
||||
process_latency = ((evt.logMonoTime/1e9) - (evta.idx.timestampEof/1e9))*1000
|
||||
|
||||
# put in header (first)
|
||||
if not seen_iframe:
|
||||
if nvidia:
|
||||
nvDec.DecodeSurfaceFromPacket(np.frombuffer(evta.header, dtype=np.uint8))
|
||||
else:
|
||||
codec.decode(av.packet.Packet(evta.header))
|
||||
seen_iframe = True
|
||||
|
||||
if nvidia:
|
||||
rawSurface = nvDec.DecodeSurfaceFromPacket(np.frombuffer(evta.data, dtype=np.uint8))
|
||||
if rawSurface.Empty():
|
||||
if debug:
|
||||
print("DROP SURFACE")
|
||||
continue
|
||||
convSurface = conv_yuv.Execute(rawSurface, cc1)
|
||||
nvDwn_yuv.DownloadSingleSurface(convSurface, img_yuv)
|
||||
else:
|
||||
frames = codec.decode(av.packet.Packet(evta.data))
|
||||
if len(frames) == 0:
|
||||
if debug:
|
||||
print("DROP SURFACE")
|
||||
continue
|
||||
assert len(frames) == 1
|
||||
img_yuv = frames[0].to_ndarray(format=av.video.format.VideoFormat('yuv420p')).flatten()
|
||||
uv_offset = H*W
|
||||
y = img_yuv[:uv_offset]
|
||||
uv = img_yuv[uv_offset:].reshape(2, -1).ravel('F')
|
||||
img_yuv = np.hstack((y, uv))
|
||||
|
||||
vipc_server.send(vst, img_yuv.data, cnt, int(time_q[0]*1e9), int(time.monotonic()*1e9))
|
||||
cnt += 1
|
||||
|
||||
pc_latency = (time.monotonic()-time_q[0])*1000
|
||||
time_q = time_q[1:]
|
||||
if debug:
|
||||
print("%2d %4d %.3f %.3f roll %6.2f ms latency %6.2f ms + %6.2f ms + %6.2f ms = %6.2f ms"
|
||||
% (len(msgs), evta.idx.encodeId, evt.logMonoTime/1e9, evta.idx.timestampEof/1e6, frame_latency,
|
||||
process_latency, network_latency, pc_latency, process_latency+network_latency+pc_latency ), len(evta.data), sock_name)
|
||||
|
||||
class CompressedVipc:
|
||||
def __init__(self, addr, vision_streams, nvidia=False, debug=False):
|
||||
self.vipc_server = VisionIpcServer("camerad")
|
||||
for vst in vision_streams:
|
||||
self.vipc_server.create_buffers(vst, 4, False, W, H)
|
||||
self.vipc_server.start_listener()
|
||||
|
||||
self.procs = []
|
||||
for vst in vision_streams:
|
||||
p = multiprocessing.Process(target=decoder, args=(addr, self.vipc_server, vst, nvidia, debug))
|
||||
p.start()
|
||||
self.procs.append(p)
|
||||
|
||||
def join(self):
|
||||
for p in self.procs:
|
||||
p.join()
|
||||
|
||||
def kill(self):
|
||||
for p in self.procs:
|
||||
p.terminate()
|
||||
self.join()
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Decode video streams and broadcast on VisionIPC")
|
||||
parser.add_argument("addr", help="Address of comma three")
|
||||
parser.add_argument("--nvidia", action="store_true", help="Use nvidia instead of ffmpeg")
|
||||
parser.add_argument("--cams", default="0,1,2", help="Cameras to decode")
|
||||
parser.add_argument("--silent", action="store_true", help="Suppress debug output")
|
||||
args = parser.parse_args()
|
||||
|
||||
vision_streams = [
|
||||
VisionStreamType.VISION_STREAM_ROAD,
|
||||
VisionStreamType.VISION_STREAM_WIDE_ROAD,
|
||||
VisionStreamType.VISION_STREAM_DRIVER,
|
||||
]
|
||||
|
||||
vsts = [vision_streams[int(x)] for x in args.cams.split(",")]
|
||||
cvipc = CompressedVipc(args.addr, vsts, args.nvidia, debug=(not args.silent))
|
||||
cvipc.join()
|
||||
44
tools/camerastream/receive.py
Executable file
44
tools/camerastream/receive.py
Executable file
@@ -0,0 +1,44 @@
|
||||
#!/usr/bin/env python
|
||||
import os
|
||||
import sys
|
||||
import numpy as np
|
||||
os.environ['ZMQ'] = '1'
|
||||
|
||||
from openpilot.common.window import Window
|
||||
import cereal.messaging as messaging
|
||||
|
||||
# start camerad with 'SEND_ROAD=1 SEND_DRIVER=1 SEND_WIDE_ROAD=1 XMIN=771 XMAX=1156 YMIN=483 YMAX=724 ./camerad'
|
||||
# also start bridge
|
||||
# then run this "./receive.py <ip>"
|
||||
|
||||
if "FULL" in os.environ:
|
||||
SCALE = 2
|
||||
XMIN, XMAX = 0, 1927
|
||||
YMIN, YMAX = 0, 1207
|
||||
else:
|
||||
SCALE = 1
|
||||
XMIN = 771
|
||||
XMAX = 1156
|
||||
YMIN = 483
|
||||
YMAX = 724
|
||||
H, W = ((YMAX-YMIN+1)//SCALE, (XMAX-XMIN+1)//SCALE)
|
||||
|
||||
if __name__ == '__main__':
|
||||
cameras = ['roadCameraState', 'wideRoadCameraState', 'driverCameraState']
|
||||
if "CAM" in os.environ:
|
||||
cam = int(os.environ['CAM'])
|
||||
cameras = cameras[cam:cam+1]
|
||||
sm = messaging.SubMaster(cameras, addr=sys.argv[1])
|
||||
win = Window(W*len(cameras), H)
|
||||
bdat = np.zeros((H, W*len(cameras), 3), dtype=np.uint8)
|
||||
|
||||
while 1:
|
||||
sm.update()
|
||||
for i,k in enumerate(cameras):
|
||||
if sm.updated[k]:
|
||||
#print("update", k)
|
||||
bgr_raw = sm[k].image
|
||||
dat = np.frombuffer(bgr_raw, dtype=np.uint8).reshape(H, W, 3)[:, :, [2,1,0]]
|
||||
bdat[:, W*i:W*(i+1)] = dat
|
||||
win.draw(bdat)
|
||||
|
||||
33
tools/gpstest/README.md
Normal file
33
tools/gpstest/README.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# GPS test setup
|
||||
Testing the GPS receiver using GPS spoofing. At the moment only
|
||||
static location relpay is supported.
|
||||
|
||||
# Usage
|
||||
on C3 run `rpc_server.py`, on host PC run `fuzzy_testing.py`
|
||||
|
||||
`simulate_gps_signal.py` downloads the latest ephemeris file from
|
||||
https://cddis.nasa.gov/archive/gnss/data/daily/20xx/brdc/.
|
||||
|
||||
|
||||
# Hardware Setup
|
||||
* [LimeSDR USB](https://wiki.myriadrf.org/LimeSDR-USB)
|
||||
* Asus AX58BT antenna
|
||||
|
||||
# Software Setup
|
||||
* https://github.com/myriadrf/LimeSuite
|
||||
To communicate with LimeSDR the LimeSuite is needed it abstracts the direct
|
||||
communication. It also contains examples for a quick start.
|
||||
|
||||
The latest stable version (22.09) does not have the corresponding firmware
|
||||
download available at https://downloads.myriadrf.org/project/limesuite. Therefore
|
||||
version 20.10 was chosen.
|
||||
|
||||
* https://github.com/osqzss/LimeGPS
|
||||
Built on top of LimeSuite (libLimeSuite.so.20.10-1), generates the GPS signal.
|
||||
|
||||
```
|
||||
./LimeGPS -e <ephemeris file> -l <location coordinates>
|
||||
|
||||
# Example
|
||||
./LimeGPS -e /pathTo/brdc2660.22n -l 47.202028,15.740394,100
|
||||
```
|
||||
115
tools/gpstest/fuzzy_testing.py
Executable file
115
tools/gpstest/fuzzy_testing.py
Executable file
@@ -0,0 +1,115 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import multiprocessing
|
||||
import rpyc # pylint: disable=import-error
|
||||
from collections import defaultdict
|
||||
|
||||
from helper import download_rinex, exec_LimeGPS_bin
|
||||
from helper import get_random_coords, get_continuous_coords
|
||||
|
||||
#------------------------------------------------------------------------------
|
||||
# this script is supposed to run on HOST PC
|
||||
# limeSDR is unreliable via c3 USB
|
||||
#------------------------------------------------------------------------------
|
||||
|
||||
|
||||
def run_lime_gps(rinex_file: str, location: str, timeout: int):
|
||||
# needs to run longer than the checker
|
||||
timeout += 10
|
||||
print(f"LimeGPS {location} {timeout}")
|
||||
p = multiprocessing.Process(target=exec_LimeGPS_bin,
|
||||
args=(rinex_file, location, timeout))
|
||||
p.start()
|
||||
return p
|
||||
|
||||
con = None
|
||||
def run_remote_checker(lat, lon, alt, duration, ip_addr):
|
||||
global con
|
||||
try:
|
||||
con = rpyc.connect(ip_addr, 18861)
|
||||
con._config['sync_request_timeout'] = duration+20
|
||||
except ConnectionRefusedError:
|
||||
print("could not run remote checker is 'rpc_server.py' running???")
|
||||
return False, None, None
|
||||
|
||||
matched, log, info = con.root.exposed_run_checker(lat, lon, alt,
|
||||
timeout=duration,
|
||||
use_laikad=True)
|
||||
con.close() # TODO: might wanna fetch more logs here
|
||||
con = None
|
||||
|
||||
print(f"Remote Checker: {log} {info}")
|
||||
return matched, log, info
|
||||
|
||||
|
||||
stats = defaultdict(int) # type: ignore
|
||||
keys = ['success', 'failed', 'ublox_fail', 'laikad_fail', 'proc_crash', 'checker_crash']
|
||||
|
||||
def print_report():
|
||||
print("\nFuzzy testing report summary:")
|
||||
for k in keys:
|
||||
print(f" {k}: {stats[k]}")
|
||||
|
||||
|
||||
def update_stats(matched, log, info):
|
||||
if matched:
|
||||
stats['success'] += 1
|
||||
return
|
||||
|
||||
stats['failed'] += 1
|
||||
if log == "PROC CRASH":
|
||||
stats['proc_crash'] += 1
|
||||
if log == "CHECKER CRASHED":
|
||||
stats['checker_crash'] += 1
|
||||
if log == "TIMEOUT":
|
||||
if "LAIKAD" in info:
|
||||
stats['laikad_fail'] += 1
|
||||
else: # "UBLOX" in info
|
||||
stats['ublox_fail'] += 1
|
||||
|
||||
|
||||
def main(ip_addr, continuous_mode, timeout, pos):
|
||||
rinex_file = download_rinex()
|
||||
|
||||
lat, lon, alt = pos
|
||||
if lat == 0 and lon == 0 and alt == 0:
|
||||
lat, lon, alt = get_random_coords(47.2020, 15.7403)
|
||||
|
||||
try:
|
||||
while True:
|
||||
# spoof random location
|
||||
spoof_proc = run_lime_gps(rinex_file, f"{lat},{lon},{alt}", timeout)
|
||||
|
||||
# remote checker execs blocking
|
||||
matched, log, info = run_remote_checker(lat, lon, alt, timeout, ip_addr)
|
||||
update_stats(matched, log, info)
|
||||
spoof_proc.terminate()
|
||||
spoof_proc = None
|
||||
|
||||
if continuous_mode:
|
||||
lat, lon, alt = get_continuous_coords(lat, lon, alt)
|
||||
else:
|
||||
lat, lon, alt = get_random_coords(lat, lon)
|
||||
except KeyboardInterrupt:
|
||||
if spoof_proc is not None:
|
||||
spoof_proc.terminate()
|
||||
|
||||
if con is not None and not con.closed:
|
||||
con.root.exposed_kill_procs()
|
||||
con.close()
|
||||
|
||||
print_report()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Fuzzy test GPS stack with random locations.")
|
||||
parser.add_argument("ip_addr", type=str)
|
||||
parser.add_argument("-c", "--contin", type=bool, nargs='?', default=False, help='Continous location change')
|
||||
parser.add_argument("-t", "--timeout", type=int, nargs='?', default=180, help='Timeout to get location')
|
||||
|
||||
# for replaying a location
|
||||
parser.add_argument("lat", type=float, nargs='?', default=0)
|
||||
parser.add_argument("lon", type=float, nargs='?', default=0)
|
||||
parser.add_argument("alt", type=float, nargs='?', default=0)
|
||||
args = parser.parse_args()
|
||||
main(args.ip_addr, args.contin, args.timeout, (args.lat, args.lon, args.alt))
|
||||
53
tools/gpstest/helper.py
Normal file
53
tools/gpstest/helper.py
Normal file
@@ -0,0 +1,53 @@
|
||||
import random
|
||||
import datetime as dt
|
||||
import subprocess as sp
|
||||
from typing import Tuple
|
||||
|
||||
from laika.downloader import download_nav
|
||||
from laika.gps_time import GPSTime
|
||||
from laika.helpers import ConstellationId
|
||||
|
||||
|
||||
def download_rinex():
|
||||
# TODO: check if there is a better way to get the full brdc file for LimeGPS
|
||||
gps_time = GPSTime.from_datetime(dt.datetime.utcnow())
|
||||
utc_time = dt.datetime.utcnow() - dt.timedelta(1)
|
||||
gps_time = GPSTime.from_datetime(dt.datetime(utc_time.year, utc_time.month, utc_time.day))
|
||||
return download_nav(gps_time, '/tmp/gpstest/', ConstellationId.GPS)
|
||||
|
||||
|
||||
def exec_LimeGPS_bin(rinex_file: str, location: str, duration: int):
|
||||
# this functions should never return, cause return means, timeout is
|
||||
# reached or it crashed
|
||||
try:
|
||||
cmd = ["LimeGPS/LimeGPS", "-e", rinex_file, "-l", location]
|
||||
sp.check_output(cmd, timeout=duration)
|
||||
except sp.TimeoutExpired:
|
||||
print("LimeGPS timeout reached!")
|
||||
except Exception as e:
|
||||
print(f"LimeGPS crashed: {str(e)}")
|
||||
|
||||
|
||||
def get_random_coords(lat, lon) -> Tuple[float, float, int]:
|
||||
# jump around the world
|
||||
# max values, lat: -90 to 90, lon: -180 to 180
|
||||
|
||||
lat_add = random.random()*20 + 10
|
||||
lon_add = random.random()*20 + 20
|
||||
alt = random.randint(-10**3, 4*10**3)
|
||||
|
||||
lat = ((lat + lat_add + 90) % 180) - 90
|
||||
lon = ((lon + lon_add + 180) % 360) - 180
|
||||
return round(lat, 5), round(lon, 5), alt
|
||||
|
||||
|
||||
def get_continuous_coords(lat, lon, alt) -> Tuple[float, float, int]:
|
||||
# continuously move around the world
|
||||
lat_add = random.random()*0.01
|
||||
lon_add = random.random()*0.01
|
||||
alt_add = random.randint(-100, 100)
|
||||
|
||||
lat = ((lat + lat_add + 90) % 180) - 90
|
||||
lon = ((lon + lon_add + 180) % 360) - 180
|
||||
alt += alt_add
|
||||
return round(lat, 5), round(lon, 5), alt
|
||||
44
tools/gpstest/patches/hackrf.patch
Normal file
44
tools/gpstest/patches/hackrf.patch
Normal file
@@ -0,0 +1,44 @@
|
||||
diff --git a/host/hackrf-tools/src/CMakeLists.txt b/host/hackrf-tools/src/CMakeLists.txt
|
||||
index 7115151c..a51388ba 100644
|
||||
--- a/host/hackrf-tools/src/CMakeLists.txt
|
||||
+++ b/host/hackrf-tools/src/CMakeLists.txt
|
||||
@@ -23,20 +23,20 @@
|
||||
|
||||
set(INSTALL_DEFAULT_BINDIR "bin" CACHE STRING "Appended to CMAKE_INSTALL_PREFIX")
|
||||
|
||||
-find_package(FFTW REQUIRED)
|
||||
-include_directories(${FFTW_INCLUDES})
|
||||
-get_filename_component(FFTW_LIBRARY_DIRS ${FFTW_LIBRARIES} DIRECTORY)
|
||||
-link_directories(${FFTW_LIBRARY_DIRS})
|
||||
+#find_package(FFTW REQUIRED)
|
||||
+#include_directories(${FFTW_INCLUDES})
|
||||
+#get_filename_component(FFTW_LIBRARY_DIRS ${FFTW_LIBRARIES} DIRECTORY)
|
||||
+#link_directories(${FFTW_LIBRARY_DIRS})
|
||||
|
||||
SET(TOOLS
|
||||
hackrf_transfer
|
||||
- hackrf_spiflash
|
||||
- hackrf_cpldjtag
|
||||
+ #hackrf_spiflash
|
||||
+ #hackrf_cpldjtag
|
||||
hackrf_info
|
||||
- hackrf_debug
|
||||
- hackrf_clock
|
||||
- hackrf_sweep
|
||||
- hackrf_operacake
|
||||
+ #hackrf_debug
|
||||
+ #hackrf_clock
|
||||
+ #hackrf_sweep
|
||||
+ #hackrf_operacake
|
||||
)
|
||||
|
||||
if(MSVC)
|
||||
@@ -45,7 +45,7 @@ if(MSVC)
|
||||
)
|
||||
LIST(APPEND TOOLS_LINK_LIBS ${FFTW_LIBRARIES})
|
||||
else()
|
||||
- LIST(APPEND TOOLS_LINK_LIBS m fftw3f)
|
||||
+ LIST(APPEND TOOLS_LINK_LIBS m)# fftw3f)
|
||||
endif()
|
||||
|
||||
if(NOT libhackrf_SOURCE_DIR)
|
||||
13
tools/gpstest/patches/limeGPS/inc_ephem_array_size.patch
Normal file
13
tools/gpstest/patches/limeGPS/inc_ephem_array_size.patch
Normal file
@@ -0,0 +1,13 @@
|
||||
diff --git a/gpssim.h b/gpssim.h
|
||||
index c30b227..2ae0802 100644
|
||||
--- a/gpssim.h
|
||||
+++ b/gpssim.h
|
||||
@@ -75,7 +75,7 @@
|
||||
#define SC08 (8)
|
||||
#define SC16 (16)
|
||||
|
||||
-#define EPHEM_ARRAY_SIZE (13) // for daily GPS broadcast ephemers file (brdc)
|
||||
+#define EPHEM_ARRAY_SIZE (20) // for daily GPS broadcast ephemers file (brdc)
|
||||
|
||||
/*! \brief Structure representing GPS time */
|
||||
typedef struct
|
||||
11
tools/gpstest/patches/limeGPS/makefile.patch
Normal file
11
tools/gpstest/patches/limeGPS/makefile.patch
Normal file
@@ -0,0 +1,11 @@
|
||||
diff --git a/makefile b/makefile
|
||||
index 51bfabf..d0ea1eb 100644
|
||||
--- a/makefile
|
||||
+++ b/makefile
|
||||
@@ -1,5 +1,4 @@
|
||||
CC=gcc -O2 -Wall
|
||||
|
||||
all: limegps.c gpssim.c
|
||||
- $(CC) -o LimeGPS limegps.c gpssim.c -lm -lpthread -lLimeSuite
|
||||
-
|
||||
+ $(CC) -o LimeGPS limegps.c gpssim.c -lm -lpthread -lLimeSuite -I../LimeSuite/src -L../LimeSuite/builddir/src -Wl,-rpath="$(PWD)/../LimeSuite/builddir/src"
|
||||
13
tools/gpstest/patches/limeSuite/mcu_error.patch
Normal file
13
tools/gpstest/patches/limeSuite/mcu_error.patch
Normal file
@@ -0,0 +1,13 @@
|
||||
diff --git a/src/lms7002m/LMS7002M_RxTxCalibrations.cpp b/src/lms7002m/LMS7002M_RxTxCalibrations.cpp
|
||||
index 41a37044..ac29c6b6 100644
|
||||
--- a/src/lms7002m/LMS7002M_RxTxCalibrations.cpp
|
||||
+++ b/src/lms7002m/LMS7002M_RxTxCalibrations.cpp
|
||||
@@ -254,7 +254,7 @@ int LMS7002M::CalibrateTx(float_type bandwidth_Hz, bool useExtLoopback)
|
||||
mcuControl->RunProcedure(useExtLoopback ? MCU_FUNCTION_CALIBRATE_TX_EXTLOOPB : MCU_FUNCTION_CALIBRATE_TX);
|
||||
status = mcuControl->WaitForMCU(1000);
|
||||
if(status != MCU_BD::MCU_NO_ERROR)
|
||||
- return ReportError(EINVAL, "Tx Calibration: MCU error %i (%s)", status, MCU_BD::MCUStatusMessage(status));
|
||||
+ return -1; //ReportError(EINVAL, "Tx Calibration: MCU error %i (%s)", status, MCU_BD::MCUStatusMessage(status));
|
||||
}
|
||||
|
||||
//sync registers to cache
|
||||
13
tools/gpstest/patches/limeSuite/reference_print.patch
Normal file
13
tools/gpstest/patches/limeSuite/reference_print.patch
Normal file
@@ -0,0 +1,13 @@
|
||||
diff --git a/src/FPGA_common/FPGA_common.cpp b/src/FPGA_common/FPGA_common.cpp
|
||||
index 4e81f33e..7381c475 100644
|
||||
--- a/src/FPGA_common/FPGA_common.cpp
|
||||
+++ b/src/FPGA_common/FPGA_common.cpp
|
||||
@@ -946,7 +946,7 @@ double FPGA::DetectRefClk(double fx3Clk)
|
||||
|
||||
if (i == 0)
|
||||
return -1;
|
||||
- lime::info("Reference clock %1.2f MHz", clkTbl[i - 1] / 1e6);
|
||||
+ //lime::info("Reference clock %1.2f MHz", clkTbl[i - 1] / 1e6);
|
||||
return clkTbl[i - 1];
|
||||
}
|
||||
|
||||
185
tools/gpstest/rpc_server.py
Normal file
185
tools/gpstest/rpc_server.py
Normal file
@@ -0,0 +1,185 @@
|
||||
import os
|
||||
import time
|
||||
import shutil
|
||||
from datetime import datetime
|
||||
from collections import defaultdict
|
||||
|
||||
import rpyc # pylint: disable=import-error
|
||||
from rpyc.utils.server import ThreadedServer # pylint: disable=import-error
|
||||
|
||||
#from openpilot.common.params import Params
|
||||
import cereal.messaging as messaging
|
||||
from openpilot.selfdrive.manager.process_config import managed_processes
|
||||
from laika.lib.coordinates import ecef2geodetic
|
||||
|
||||
DELTA = 0.001
|
||||
ALT_DELTA = 30
|
||||
MATCH_NUM = 10
|
||||
REPORT_STATS = 10
|
||||
|
||||
EPHEM_CACHE = "/data/params/d/LaikadEphemerisV3"
|
||||
DOWNLOAD_CACHE = "/tmp/comma_download_cache"
|
||||
|
||||
SERVER_LOG_FILE = "/tmp/fuzzy_server.log"
|
||||
server_log = open(SERVER_LOG_FILE, "w+")
|
||||
|
||||
def slog(msg):
|
||||
server_log.write(f"{datetime.now().strftime('%H:%M:%S.%f')} | {msg}\n")
|
||||
server_log.flush()
|
||||
|
||||
def handle_laikad(msg):
|
||||
if not hasattr(msg, 'correctedMeasurements'):
|
||||
return None
|
||||
|
||||
num_corr = len(msg.correctedMeasurements)
|
||||
pos_ecef = msg.positionECEF.value
|
||||
pos_geo = []
|
||||
if len(pos_ecef) > 0:
|
||||
pos_geo = ecef2geodetic(pos_ecef)
|
||||
|
||||
pos_std = msg.positionECEF.std
|
||||
pos_valid = msg.positionECEF.valid
|
||||
|
||||
slog(f"{num_corr} {pos_geo} {pos_ecef} {pos_std} {pos_valid}")
|
||||
return pos_geo, (num_corr, pos_geo, list(pos_ecef), list(msg.positionECEF.std))
|
||||
|
||||
hw_msgs = 0
|
||||
ephem_msgs: dict = defaultdict(int)
|
||||
def handle_ublox(msg):
|
||||
global hw_msgs
|
||||
|
||||
d = msg.to_dict()
|
||||
|
||||
if 'hwStatus2' in d:
|
||||
hw_msgs += 1
|
||||
|
||||
if 'ephemeris' in d:
|
||||
ephem_msgs[msg.ephemeris.svId] += 1
|
||||
|
||||
num_meas = None
|
||||
if 'measurementReport' in d:
|
||||
num_meas = msg.measurementReport.numMeas
|
||||
|
||||
return [hw_msgs, ephem_msgs, num_meas]
|
||||
|
||||
|
||||
def start_procs(procs):
|
||||
for p in procs:
|
||||
managed_processes[p].start()
|
||||
time.sleep(1)
|
||||
|
||||
def kill_procs(procs, no_retry=False):
|
||||
for p in procs:
|
||||
managed_processes[p].stop()
|
||||
time.sleep(1)
|
||||
|
||||
if not no_retry:
|
||||
for p in procs:
|
||||
mp = managed_processes[p].proc
|
||||
if mp is not None and mp.is_alive():
|
||||
managed_processes[p].stop()
|
||||
time.sleep(3)
|
||||
|
||||
def check_alive_procs(procs):
|
||||
for p in procs:
|
||||
mp = managed_processes[p].proc
|
||||
if mp is None or not mp.is_alive():
|
||||
return False, p
|
||||
return True, None
|
||||
|
||||
|
||||
class RemoteCheckerService(rpyc.Service):
|
||||
def on_connect(self, conn):
|
||||
pass
|
||||
|
||||
def on_disconnect(self, conn):
|
||||
#kill_procs(self.procs, no_retry=False)
|
||||
# this execution is delayed, it will kill the next run of laikad
|
||||
# TODO: add polling to wait for everything is killed
|
||||
pass
|
||||
|
||||
def run_checker(self, slat, slon, salt, sockets, procs, timeout):
|
||||
global hw_msgs, ephem_msgs
|
||||
hw_msgs = 0
|
||||
ephem_msgs = defaultdict(int)
|
||||
|
||||
slog(f"Run test: {slat} {slon} {salt}")
|
||||
|
||||
# quectel_mod = Params().get_bool("UbloxAvailable")
|
||||
|
||||
match_cnt = 0
|
||||
msg_cnt = 0
|
||||
stats_laikad = []
|
||||
stats_ublox = []
|
||||
|
||||
self.procs = procs
|
||||
start_procs(procs)
|
||||
sm = messaging.SubMaster(sockets)
|
||||
|
||||
start_time = time.monotonic()
|
||||
while True:
|
||||
sm.update()
|
||||
|
||||
if sm.updated['ubloxGnss']:
|
||||
stats_ublox.append(handle_ublox(sm['ubloxGnss']))
|
||||
|
||||
if sm.updated['gnssMeasurements']:
|
||||
pos_geo, stats = handle_laikad(sm['gnssMeasurements'])
|
||||
if pos_geo is None or len(pos_geo) == 0:
|
||||
continue
|
||||
|
||||
match = all(abs(g-s) < DELTA for g,s in zip(pos_geo[:2], [slat, slon], strict=True))
|
||||
match &= abs(pos_geo[2] - salt) < ALT_DELTA
|
||||
if match:
|
||||
match_cnt += 1
|
||||
if match_cnt >= MATCH_NUM:
|
||||
return True, "MATCH", f"After: {round(time.monotonic() - start_time, 4)}"
|
||||
|
||||
# keep some stats for error reporting
|
||||
stats_laikad.append(stats)
|
||||
|
||||
if (msg_cnt % 10) == 0:
|
||||
a, p = check_alive_procs(procs)
|
||||
if not a:
|
||||
return False, "PROC CRASH", f"{p}"
|
||||
msg_cnt += 1
|
||||
|
||||
if (time.monotonic() - start_time) > timeout:
|
||||
h = f"LAIKAD: {stats_laikad[-REPORT_STATS:]}"
|
||||
if len(h) == 0:
|
||||
h = f"UBLOX: {stats_ublox[-REPORT_STATS:]}"
|
||||
return False, "TIMEOUT", h
|
||||
|
||||
|
||||
def exposed_run_checker(self, slat, slon, salt, timeout=180, use_laikad=True):
|
||||
try:
|
||||
procs = []
|
||||
sockets = []
|
||||
|
||||
if use_laikad:
|
||||
procs.append("laikad") # pigeond, ubloxd # might wanna keep them running
|
||||
sockets += ['ubloxGnss', 'gnssMeasurements']
|
||||
|
||||
if os.path.exists(EPHEM_CACHE):
|
||||
os.remove(EPHEM_CACHE)
|
||||
shutil.rmtree(DOWNLOAD_CACHE, ignore_errors=True)
|
||||
|
||||
ret = self.run_checker(slat, slon, salt, sockets, procs, timeout)
|
||||
kill_procs(procs)
|
||||
return ret
|
||||
|
||||
except Exception as e:
|
||||
# always make sure processes get killed
|
||||
kill_procs(procs)
|
||||
return False, "CHECKER CRASHED", f"{str(e)}"
|
||||
|
||||
|
||||
def exposed_kill_procs(self):
|
||||
kill_procs(self.procs, no_retry=True)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
print(f"Sever Log written to: {SERVER_LOG_FILE}")
|
||||
t = ThreadedServer(RemoteCheckerService, port=18861)
|
||||
t.start()
|
||||
|
||||
16
tools/gpstest/run_unittest.sh
Executable file
16
tools/gpstest/run_unittest.sh
Executable file
@@ -0,0 +1,16 @@
|
||||
#!/bin/bash
|
||||
|
||||
# NOTE: can only run inside limeGPS test box!
|
||||
|
||||
# run limeGPS with random static location
|
||||
timeout 300 ./simulate_gps_signal.py 32.7518 -117.1962 &
|
||||
gps_PID=$(ps -aux | grep -m 1 "timeout 300" | awk '{print $2}')
|
||||
|
||||
echo "starting limeGPS..."
|
||||
sleep 10
|
||||
|
||||
# run unit tests (skipped when module not present)
|
||||
python -m unittest test_gps.py
|
||||
python -m unittest test_gps_qcom.py
|
||||
|
||||
kill $gps_PID
|
||||
25
tools/gpstest/setup.sh
Executable file
25
tools/gpstest/setup.sh
Executable file
@@ -0,0 +1,25 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
cd $DIR
|
||||
|
||||
if [ ! -d LimeSuite ]; then
|
||||
git clone https://github.com/myriadrf/LimeSuite.git
|
||||
cd LimeSuite
|
||||
# checkout latest version which has firmware updates available
|
||||
git checkout v20.10.0
|
||||
git apply ../patches/limeSuite/*
|
||||
mkdir builddir && cd builddir
|
||||
cmake -DCMAKE_BUILD_TYPE=Release ..
|
||||
make -j4
|
||||
cd ../..
|
||||
fi
|
||||
|
||||
if [ ! -d LimeGPS ]; then
|
||||
git clone https://github.com/osqzss/LimeGPS.git
|
||||
cd LimeGPS
|
||||
git apply ../patches/limeGPS/*
|
||||
make
|
||||
cd ..
|
||||
fi
|
||||
21
tools/gpstest/setup_hackrf.sh
Executable file
21
tools/gpstest/setup_hackrf.sh
Executable file
@@ -0,0 +1,21 @@
|
||||
#!/bin/bash
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
cd $DIR
|
||||
|
||||
if [ ! -d gps-sdr-sim ]; then
|
||||
git clone https://github.com/osqzss/gps-sdr-sim.git
|
||||
cd gps-sdr-sim
|
||||
make
|
||||
cd ..
|
||||
fi
|
||||
|
||||
if [ ! -d hackrf ]; then
|
||||
git clone https://github.com/greatscottgadgets/hackrf.git
|
||||
cd hackrf/host
|
||||
git apply ../../patches/hackrf.patch
|
||||
cmake .
|
||||
make
|
||||
fi
|
||||
|
||||
151
tools/gpstest/simulate_gps_signal.py
Executable file
151
tools/gpstest/simulate_gps_signal.py
Executable file
@@ -0,0 +1,151 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import random
|
||||
import argparse
|
||||
import datetime as dt
|
||||
import subprocess as sp
|
||||
from typing import Tuple
|
||||
|
||||
from laika.downloader import download_nav
|
||||
from laika.gps_time import GPSTime
|
||||
from laika.helpers import ConstellationId
|
||||
|
||||
cache_dir = '/tmp/gpstest/'
|
||||
|
||||
|
||||
def download_rinex():
|
||||
# TODO: check if there is a better way to get the full brdc file for LimeGPS
|
||||
gps_time = GPSTime.from_datetime(dt.datetime.utcnow())
|
||||
utc_time = dt.datetime.utcnow()# - dt.timedelta(1)
|
||||
gps_time = GPSTime.from_datetime(dt.datetime(utc_time.year, utc_time.month, utc_time.day))
|
||||
return download_nav(gps_time, cache_dir, ConstellationId.GPS)
|
||||
|
||||
def get_coords(lat, lon, s1, s2, o1=0, o2=0) -> Tuple[int, int]:
|
||||
lat_add = random.random()*s1 + o1
|
||||
lon_add = random.random()*s2 + o2
|
||||
|
||||
lat = ((lat + lat_add + 90) % 180) - 90
|
||||
lon = ((lon + lon_add + 180) % 360) - 180
|
||||
return round(lat, 5), round(lon, 5)
|
||||
|
||||
def get_continuous_coords(lat, lon) -> Tuple[int, int]:
|
||||
# continuously move around the world
|
||||
return get_coords(lat, lon, 0.01, 0.01)
|
||||
|
||||
def get_random_coords(lat, lon) -> Tuple[int, int]:
|
||||
# jump around the world
|
||||
return get_coords(lat, lon, 20, 20, 10, 20)
|
||||
|
||||
def run_limeSDR_loop(lat, lon, alt, contin_sim, rinex_file, timeout):
|
||||
while True:
|
||||
try:
|
||||
# TODO: add starttime setting and altitude
|
||||
# -t 2023/01/15,00:00:00 -T 2023/01/15,00:00:00
|
||||
# this needs to match the date of the navigation file
|
||||
print(f"starting LimeGPS, Location: {lat} {lon} {alt}")
|
||||
cmd = ["LimeGPS/LimeGPS", "-e", rinex_file, "-l", f"{lat},{lon},{alt}"]
|
||||
print(f"CMD: {cmd}")
|
||||
sp.check_output(cmd, stderr=sp.PIPE, timeout=timeout)
|
||||
except KeyboardInterrupt:
|
||||
print("stopping LimeGPS")
|
||||
return
|
||||
except sp.TimeoutExpired:
|
||||
print("LimeGPS timeout reached!")
|
||||
except Exception as e:
|
||||
out_stderr = e.stderr.decode('utf-8')# pylint:disable=no-member
|
||||
if "Device is busy." in out_stderr:
|
||||
print("GPS simulation is already running, Device is busy!")
|
||||
return
|
||||
|
||||
print(f"LimeGPS crashed: {str(e)}")
|
||||
print(f"stderr:\n{e.stderr.decode('utf-8')}")# pylint:disable=no-member
|
||||
return
|
||||
|
||||
if contin_sim:
|
||||
lat, lon = get_continuous_coords(lat, lon)
|
||||
else:
|
||||
lat, lon = get_random_coords(lat, lon)
|
||||
|
||||
def run_hackRF_loop(lat, lon, rinex_file, timeout):
|
||||
|
||||
if timeout is not None:
|
||||
print("no jump mode for hackrf!")
|
||||
return
|
||||
|
||||
try:
|
||||
print(f"starting gps-sdr-sim, Location: {lat},{lon}")
|
||||
# create 30second file and replay with hackrf endless
|
||||
cmd = ["gps-sdr-sim/gps-sdr-sim", "-e", rinex_file, "-l", f"{lat},{lon},-200", "-d", "30"]
|
||||
sp.check_output(cmd, stderr=sp.PIPE, timeout=timeout)
|
||||
# created in current working directory
|
||||
except Exception:
|
||||
print("Failed to generate gpssim.bin")
|
||||
|
||||
try:
|
||||
print("starting hackrf_transfer")
|
||||
# create 30second file and replay with hackrf endless
|
||||
cmd = ["hackrf/host/hackrf-tools/src/hackrf_transfer", "-t", "gpssim.bin",
|
||||
"-f", "1575420000", "-s", "2600000", "-a", "1", "-R"]
|
||||
sp.check_output(cmd, stderr=sp.PIPE, timeout=timeout)
|
||||
except KeyboardInterrupt:
|
||||
print("stopping hackrf_transfer")
|
||||
return
|
||||
except Exception as e:
|
||||
print(f"hackrf_transfer crashed:{str(e)}")
|
||||
|
||||
|
||||
def main(lat, lon, alt, jump_sim, contin_sim, hackrf_mode):
|
||||
|
||||
if hackrf_mode:
|
||||
if not os.path.exists('hackrf'):
|
||||
print("hackrf not found run 'setup_hackrf.sh' first")
|
||||
return
|
||||
|
||||
if not os.path.exists('gps-sdr-sim'):
|
||||
print("gps-sdr-sim not found run 'setup_hackrf.sh' first")
|
||||
return
|
||||
|
||||
output = sp.check_output(["hackrf/host/hackrf-tools/src/hackrf_info"])
|
||||
if output.strip() == b"" or b"No HackRF boards found." in output:
|
||||
print("No HackRF boards found!")
|
||||
return
|
||||
|
||||
else:
|
||||
if not os.path.exists('LimeGPS'):
|
||||
print("LimeGPS not found run 'setup.sh' first")
|
||||
return
|
||||
|
||||
if not os.path.exists('LimeSuite'):
|
||||
print("LimeSuite not found run 'setup.sh' first")
|
||||
return
|
||||
|
||||
output = sp.check_output(["LimeSuite/builddir/LimeUtil/LimeUtil", "--find"])
|
||||
if output.strip() == b"":
|
||||
print("No LimeSDR device found!")
|
||||
return
|
||||
print(f"Device: {output.strip().decode('utf-8')}")
|
||||
|
||||
if lat == 0 and lon == 0:
|
||||
lat, lon = get_random_coords(47.2020, 15.7403)
|
||||
|
||||
rinex_file = download_rinex()
|
||||
|
||||
timeout = None
|
||||
if jump_sim:
|
||||
timeout = 30
|
||||
|
||||
if hackrf_mode:
|
||||
run_hackRF_loop(lat, lon, rinex_file, timeout)
|
||||
else:
|
||||
run_limeSDR_loop(lat, lon, alt, contin_sim, rinex_file, timeout)
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="Simulate static [or random jumping] GPS signal.")
|
||||
parser.add_argument("lat", type=float, nargs='?', default=0)
|
||||
parser.add_argument("lon", type=float, nargs='?', default=0)
|
||||
parser.add_argument("alt", type=float, nargs='?', default=0)
|
||||
parser.add_argument("--jump", action="store_true", help="signal that jumps around the world")
|
||||
parser.add_argument("--contin", action="store_true", help="continuously/slowly moving around the world")
|
||||
parser.add_argument("--hackrf", action="store_true", help="hackrf mode (DEFAULT: LimeSDR)")
|
||||
args = parser.parse_args()
|
||||
main(args.lat, args.lon, args.alt, args.jump, args.contin, args.hackrf)
|
||||
191
tools/gpstest/test_gps.py
Normal file
191
tools/gpstest/test_gps.py
Normal file
@@ -0,0 +1,191 @@
|
||||
#!/usr/bin/env python3
|
||||
import time
|
||||
import unittest
|
||||
import struct
|
||||
|
||||
from openpilot.common.params import Params
|
||||
import cereal.messaging as messaging
|
||||
import openpilot.system.sensord.pigeond as pd
|
||||
from openpilot.system.hardware import TICI
|
||||
from openpilot.selfdrive.test.helpers import with_processes
|
||||
|
||||
|
||||
def read_events(service, duration_sec):
|
||||
service_sock = messaging.sub_sock(service, timeout=0.1)
|
||||
start_time_sec = time.monotonic()
|
||||
events = []
|
||||
while time.monotonic() - start_time_sec < duration_sec:
|
||||
events += messaging.drain_sock(service_sock)
|
||||
time.sleep(0.1)
|
||||
|
||||
assert len(events) != 0, f"No '{service}'events collected!"
|
||||
return events
|
||||
|
||||
|
||||
def create_backup(pigeon):
|
||||
# controlled GNSS stop
|
||||
pigeon.send(b"\xB5\x62\x06\x04\x04\x00\x00\x00\x08\x00\x16\x74")
|
||||
|
||||
# store almanac in flash
|
||||
pigeon.send(b"\xB5\x62\x09\x14\x04\x00\x00\x00\x00\x00\x21\xEC")
|
||||
try:
|
||||
if not pigeon.wait_for_ack(ack=pd.UBLOX_SOS_ACK, nack=pd.UBLOX_SOS_NACK):
|
||||
raise RuntimeError("Could not store almanac")
|
||||
except TimeoutError:
|
||||
pass
|
||||
|
||||
|
||||
def verify_ubloxgnss_data(socket: messaging.SubSocket, max_time: int):
|
||||
start_time = 0
|
||||
end_time = 0
|
||||
events = messaging.drain_sock(socket)
|
||||
assert len(events) != 0, "no ublxGnss measurements"
|
||||
|
||||
for event in events:
|
||||
if event.ubloxGnss.which() != "measurementReport":
|
||||
continue
|
||||
|
||||
if start_time == 0:
|
||||
start_time = event.logMonoTime
|
||||
|
||||
if event.ubloxGnss.measurementReport.numMeas != 0:
|
||||
end_time = event.logMonoTime
|
||||
break
|
||||
|
||||
assert end_time != 0, "no ublox measurements received!"
|
||||
|
||||
ttfm = (end_time - start_time)/1e9
|
||||
assert ttfm < max_time, f"Time to first measurement > {max_time}s, {ttfm}"
|
||||
|
||||
# check for satellite count in measurements
|
||||
sat_count = []
|
||||
end_id = events.index(event)# pylint:disable=undefined-loop-variable
|
||||
for event in events[end_id:]:
|
||||
if event.ubloxGnss.which() == "measurementReport":
|
||||
sat_count.append(event.ubloxGnss.measurementReport.numMeas)
|
||||
|
||||
num_sat = int(sum(sat_count)/len(sat_count))
|
||||
assert num_sat >= 5, f"Not enough satellites {num_sat} (TestBox setup!)"
|
||||
|
||||
|
||||
def verify_gps_location(socket: messaging.SubSocket, max_time: int):
|
||||
events = messaging.drain_sock(socket)
|
||||
assert len(events) != 0, "no gpsLocationExternal measurements"
|
||||
|
||||
start_time = events[0].logMonoTime
|
||||
end_time = 0
|
||||
for event in events:
|
||||
gps_valid = event.gpsLocationExternal.flags % 2
|
||||
|
||||
if gps_valid:
|
||||
end_time = event.logMonoTime
|
||||
break
|
||||
|
||||
assert end_time != 0, "GPS location never converged!"
|
||||
|
||||
ttfl = (end_time - start_time)/1e9
|
||||
assert ttfl < max_time, f"Time to first location > {max_time}s, {ttfl}"
|
||||
|
||||
hacc = events[-1].gpsLocationExternal.accuracy
|
||||
vacc = events[-1].gpsLocationExternal.verticalAccuracy
|
||||
assert hacc < 20, f"Horizontal accuracy too high, {hacc}"
|
||||
assert vacc < 45, f"Vertical accuracy too high, {vacc}"
|
||||
|
||||
|
||||
def verify_time_to_first_fix(pigeon):
|
||||
# get time to first fix from nav status message
|
||||
nav_status = b""
|
||||
while True:
|
||||
pigeon.send(b"\xb5\x62\x01\x03\x00\x00\x04\x0d")
|
||||
nav_status = pigeon.receive()
|
||||
if nav_status[:4] == b"\xb5\x62\x01\x03":
|
||||
break
|
||||
|
||||
values = struct.unpack("<HHHIBBBBIIH", nav_status[:24])
|
||||
ttff = values[8]/1000
|
||||
# srms = values[9]/1000
|
||||
assert ttff < 40, f"Time to first fix > 40s, {ttff}"
|
||||
|
||||
|
||||
class TestGPS(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
if not TICI:
|
||||
raise unittest.SkipTest
|
||||
|
||||
ublox_available = Params().get_bool("UbloxAvailable")
|
||||
if not ublox_available:
|
||||
raise unittest.SkipTest
|
||||
|
||||
|
||||
def tearDown(self):
|
||||
pd.set_power(False)
|
||||
|
||||
@with_processes(['ubloxd'])
|
||||
def test_a_ublox_reset(self):
|
||||
|
||||
pigeon, pm = pd.create_pigeon()
|
||||
pd.init_baudrate(pigeon)
|
||||
assert pigeon.reset_device(), "Could not reset device!"
|
||||
|
||||
pd.initialize_pigeon(pigeon)
|
||||
|
||||
ugs = messaging.sub_sock("ubloxGnss", timeout=0.1)
|
||||
gle = messaging.sub_sock("gpsLocationExternal", timeout=0.1)
|
||||
|
||||
# receive some messages (restart after cold start takes up to 30seconds)
|
||||
pd.run_receiving(pigeon, pm, 60)
|
||||
|
||||
# store almanac for next test
|
||||
create_backup(pigeon)
|
||||
|
||||
verify_ubloxgnss_data(ugs, 60)
|
||||
verify_gps_location(gle, 60)
|
||||
|
||||
# skip for now, this might hang for a while
|
||||
#verify_time_to_first_fix(pigeon)
|
||||
|
||||
|
||||
@with_processes(['ubloxd'])
|
||||
def test_b_ublox_almanac(self):
|
||||
pigeon, pm = pd.create_pigeon()
|
||||
pd.init_baudrate(pigeon)
|
||||
|
||||
# device cold start
|
||||
pigeon.send(b"\xb5\x62\x06\x04\x04\x00\xff\xff\x00\x00\x0c\x5d")
|
||||
time.sleep(1) # wait for cold start
|
||||
pd.init_baudrate(pigeon)
|
||||
|
||||
# clear configuration
|
||||
pigeon.send_with_ack(b"\xb5\x62\x06\x09\x0d\x00\x00\x00\x1f\x1f\x00\x00\x00\x00\x00\x00\x00\x00\x17\x71\x5b")
|
||||
|
||||
# restoring almanac backup
|
||||
pigeon.send(b"\xB5\x62\x09\x14\x00\x00\x1D\x60")
|
||||
status = pigeon.wait_for_backup_restore_status()
|
||||
assert status == 2, "Could not restore almanac backup"
|
||||
|
||||
pd.initialize_pigeon(pigeon)
|
||||
|
||||
ugs = messaging.sub_sock("ubloxGnss", timeout=0.1)
|
||||
gle = messaging.sub_sock("gpsLocationExternal", timeout=0.1)
|
||||
|
||||
pd.run_receiving(pigeon, pm, 15)
|
||||
verify_ubloxgnss_data(ugs, 15)
|
||||
verify_gps_location(gle, 20)
|
||||
|
||||
|
||||
@with_processes(['ubloxd'])
|
||||
def test_c_ublox_startup(self):
|
||||
pigeon, pm = pd.create_pigeon()
|
||||
pd.init_baudrate(pigeon)
|
||||
pd.initialize_pigeon(pigeon)
|
||||
|
||||
ugs = messaging.sub_sock("ubloxGnss", timeout=0.1)
|
||||
gle = messaging.sub_sock("gpsLocationExternal", timeout=0.1)
|
||||
pd.run_receiving(pigeon, pm, 10)
|
||||
verify_ubloxgnss_data(ugs, 10)
|
||||
verify_gps_location(gle, 10)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
80
tools/gpstest/test_gps_qcom.py
Normal file
80
tools/gpstest/test_gps_qcom.py
Normal file
@@ -0,0 +1,80 @@
|
||||
#!/usr/bin/env python3
|
||||
import time
|
||||
import unittest
|
||||
import subprocess as sp
|
||||
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.system.hardware import TICI
|
||||
import cereal.messaging as messaging
|
||||
from openpilot.selfdrive.manager.process_config import managed_processes
|
||||
|
||||
|
||||
def exec_mmcli(cmd):
|
||||
cmd = "mmcli -m 0 " + cmd
|
||||
p = sp.Popen(cmd, shell=True, stdout=sp.PIPE, stderr=sp.PIPE)
|
||||
return p.communicate()
|
||||
|
||||
|
||||
def wait_for_location(socket, timeout):
|
||||
while True:
|
||||
events = messaging.drain_sock(socket)
|
||||
for event in events:
|
||||
if event.gpsLocation.flags % 2:
|
||||
return False
|
||||
|
||||
timeout -= 1
|
||||
if timeout <= 0:
|
||||
return True
|
||||
|
||||
time.sleep(0.1)
|
||||
continue
|
||||
|
||||
|
||||
class TestGPS(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
if not TICI:
|
||||
raise unittest.SkipTest
|
||||
|
||||
ublox_available = Params().get_bool("UbloxAvailable")
|
||||
if ublox_available:
|
||||
raise unittest.SkipTest
|
||||
|
||||
def test_a_quectel_cold_start(self):
|
||||
# delete assistance data to enforce cold start for GNSS
|
||||
# testing shows that this takes up to 20min
|
||||
|
||||
_, err = exec_mmcli("--command='AT+QGPSDEL=0'")
|
||||
assert len(err) == 0, f"GPSDEL failed: {err}"
|
||||
|
||||
managed_processes['rawgpsd'].start()
|
||||
start_time = time.monotonic()
|
||||
glo = messaging.sub_sock("gpsLocation", timeout=0.1)
|
||||
|
||||
timeout = 10*60*3 # 3 minute
|
||||
timedout = wait_for_location(glo, timeout)
|
||||
managed_processes['rawgpsd'].stop()
|
||||
|
||||
assert timedout is False, "Waiting for location timed out (3min)!"
|
||||
|
||||
duration = time.monotonic() - start_time
|
||||
assert duration < 60, f"Received GPS location {duration}!"
|
||||
|
||||
|
||||
def test_b_quectel_startup(self):
|
||||
managed_processes['rawgpsd'].start()
|
||||
start_time = time.monotonic()
|
||||
glo = messaging.sub_sock("gpsLocation", timeout=0.1)
|
||||
|
||||
timeout = 10*60 # 1 minute
|
||||
timedout = wait_for_location(glo, timeout)
|
||||
managed_processes['rawgpsd'].stop()
|
||||
|
||||
assert timedout is False, "Waiting for location timed out (3min)!"
|
||||
|
||||
duration = time.monotonic() - start_time
|
||||
assert duration < 60, f"Received GPS location {duration}!"
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
105
tools/gpstest/test_laikad.py
Normal file
105
tools/gpstest/test_laikad.py
Normal file
@@ -0,0 +1,105 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import time
|
||||
import unittest
|
||||
|
||||
import cereal.messaging as messaging
|
||||
import openpilot.system.sensord.pigeond as pd
|
||||
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.system.hardware import TICI
|
||||
from openpilot.selfdrive.manager.process_config import managed_processes
|
||||
from openpilot.selfdrive.test.helpers import with_processes
|
||||
|
||||
|
||||
def wait_for_location(sm, timeout, con=10):
|
||||
cons_meas = 0
|
||||
start_time = time.monotonic()
|
||||
while (time.monotonic() - start_time) < timeout:
|
||||
sm.update()
|
||||
if not sm.updated["gnssMeasurements"]:
|
||||
continue
|
||||
|
||||
msg = sm["gnssMeasurements"]
|
||||
cons_meas = (cons_meas + 1) if 'positionECEF' in msg.to_dict() else 0
|
||||
if cons_meas >= con:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
class TestLaikad(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(self):
|
||||
if not TICI:
|
||||
raise unittest.SkipTest
|
||||
|
||||
ublox_available = Params().get_bool("UbloxAvailable")
|
||||
if not ublox_available:
|
||||
raise unittest.SkipTest
|
||||
|
||||
def setUp(self):
|
||||
# ensure laikad cold start
|
||||
Params().remove("LaikadEphemerisV3")
|
||||
os.environ["LAIKAD_NO_INTERNET"] = "1"
|
||||
managed_processes['laikad'].start()
|
||||
|
||||
def tearDown(self):
|
||||
managed_processes['laikad'].stop()
|
||||
|
||||
|
||||
@with_processes(['pigeond', 'ubloxd'])
|
||||
def test_laikad_cold_start(self):
|
||||
time.sleep(5)
|
||||
|
||||
start_time = time.monotonic()
|
||||
sm = messaging.SubMaster(["gnssMeasurements"])
|
||||
|
||||
success = wait_for_location(sm, 60*2, con=10)
|
||||
duration = time.monotonic() - start_time
|
||||
|
||||
assert success, "Waiting for location timed out (2min)!"
|
||||
assert duration < 60, f"Received Location {duration}!"
|
||||
|
||||
|
||||
@with_processes(['ubloxd'])
|
||||
def test_laikad_ublox_reset_start(self):
|
||||
time.sleep(2)
|
||||
|
||||
pigeon, pm = pd.create_pigeon()
|
||||
pd.init_baudrate(pigeon)
|
||||
assert pigeon.reset_device(), "Could not reset device!"
|
||||
|
||||
laikad_sock = messaging.sub_sock("gnssMeasurements", timeout=0.1)
|
||||
ublox_gnss_sock = messaging.sub_sock("ubloxGnss", timeout=0.1)
|
||||
|
||||
pd.init_baudrate(pigeon)
|
||||
pd.initialize_pigeon(pigeon)
|
||||
pd.run_receiving(pigeon, pm, 180)
|
||||
|
||||
ublox_msgs = messaging.drain_sock(ublox_gnss_sock)
|
||||
laikad_msgs = messaging.drain_sock(laikad_sock)
|
||||
|
||||
gps_ephem_cnt = 0
|
||||
glonass_ephem_cnt = 0
|
||||
for um in ublox_msgs:
|
||||
if um.ubloxGnss.which() == 'ephemeris':
|
||||
gps_ephem_cnt += 1
|
||||
elif um.ubloxGnss.which() == 'glonassEphemeris':
|
||||
glonass_ephem_cnt += 1
|
||||
|
||||
assert gps_ephem_cnt > 0, "NO gps ephemeris collected!"
|
||||
assert glonass_ephem_cnt > 0, "NO glonass ephemeris collected!"
|
||||
|
||||
pos_meas = 0
|
||||
duration = -1
|
||||
for lm in laikad_msgs:
|
||||
pos_meas = (pos_meas + 1) if 'positionECEF' in lm.gnssMeasurements.to_dict() else 0
|
||||
if pos_meas > 5:
|
||||
duration = (lm.logMonoTime - laikad_msgs[0].logMonoTime)*1e-9
|
||||
break
|
||||
|
||||
assert pos_meas > 5, "NOT enough positions at end of read!"
|
||||
assert duration < 120, "Laikad took too long to get a Position!"
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
76
tools/install_python_dependencies.sh
Executable file
76
tools/install_python_dependencies.sh
Executable file
@@ -0,0 +1,76 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
ROOT=$DIR/../
|
||||
cd $ROOT
|
||||
|
||||
RC_FILE="${HOME}/.$(basename ${SHELL})rc"
|
||||
if [ "$(uname)" == "Darwin" ] && [ $SHELL == "/bin/bash" ]; then
|
||||
RC_FILE="$HOME/.bash_profile"
|
||||
fi
|
||||
|
||||
if ! command -v "pyenv" > /dev/null 2>&1; then
|
||||
echo "pyenv install ..."
|
||||
curl -L https://github.com/pyenv/pyenv-installer/raw/master/bin/pyenv-installer | bash
|
||||
|
||||
cat <<EOF > "${HOME}/.pyenvrc"
|
||||
if [ -z "\$PYENV_ROOT" ]; then
|
||||
export PATH=\$HOME/.pyenv/bin:\$HOME/.pyenv/shims:\$PATH
|
||||
export PYENV_ROOT="\$HOME/.pyenv"
|
||||
eval "\$(pyenv init --path)"
|
||||
eval "\$(pyenv init -)"
|
||||
eval "\$(pyenv virtualenv-init -)"
|
||||
fi
|
||||
EOF
|
||||
echo -e "\nsource ~/.pyenvrc" >> $RC_FILE
|
||||
|
||||
# activate pyenv now
|
||||
source $RC_FILE
|
||||
fi
|
||||
|
||||
export MAKEFLAGS="-j$(nproc)"
|
||||
|
||||
PYENV_PYTHON_VERSION=$(cat $ROOT/.python-version)
|
||||
if ! pyenv prefix ${PYENV_PYTHON_VERSION} &> /dev/null; then
|
||||
# no pyenv update on mac
|
||||
if [ "$(uname)" == "Linux" ]; then
|
||||
echo "pyenv update ..."
|
||||
pyenv update
|
||||
fi
|
||||
echo "python ${PYENV_PYTHON_VERSION} install ..."
|
||||
CONFIGURE_OPTS="--enable-shared" pyenv install -f ${PYENV_PYTHON_VERSION}
|
||||
fi
|
||||
eval "$(pyenv init --path)"
|
||||
|
||||
echo "update pip"
|
||||
pip install pip==23.2.1
|
||||
pip install poetry==1.5.1
|
||||
|
||||
poetry config virtualenvs.prefer-active-python true --local
|
||||
poetry config virtualenvs.in-project true --local
|
||||
|
||||
echo "PYTHONPATH=${PWD}" > $ROOT/.env
|
||||
if [[ "$(uname)" == 'Darwin' ]]; then
|
||||
echo "# msgq doesn't work on mac" >> $ROOT/.env
|
||||
echo "export ZMQ=1" >> $ROOT/.env
|
||||
echo "export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" >> $ROOT/.env
|
||||
fi
|
||||
|
||||
poetry self add poetry-dotenv-plugin@^0.1.0
|
||||
|
||||
echo "pip packages install..."
|
||||
poetry install --no-cache --no-root
|
||||
pyenv rehash
|
||||
|
||||
[ -n "$POETRY_VIRTUALENVS_CREATE" ] && RUN="" || RUN="poetry run"
|
||||
|
||||
if [ "$(uname)" != "Darwin" ]; then
|
||||
echo "pre-commit hooks install..."
|
||||
shopt -s nullglob
|
||||
for f in .pre-commit-config.yaml */.pre-commit-config.yaml; do
|
||||
if [ -e "$ROOT/$(dirname $f)/.git" ]; then
|
||||
$RUN pre-commit install -c "$f"
|
||||
fi
|
||||
done
|
||||
fi
|
||||
135
tools/install_ubuntu_dependencies.sh
Executable file
135
tools/install_ubuntu_dependencies.sh
Executable file
@@ -0,0 +1,135 @@
|
||||
#!/usr/bin/env bash
|
||||
set -e
|
||||
|
||||
SUDO=""
|
||||
|
||||
# Use sudo if not root
|
||||
if [[ ! $(id -u) -eq 0 ]]; then
|
||||
if [[ -z $(which sudo) ]]; then
|
||||
echo "Please install sudo or run as root"
|
||||
exit 1
|
||||
fi
|
||||
SUDO="sudo"
|
||||
fi
|
||||
|
||||
# Install packages present in all supported versions of Ubuntu
|
||||
function install_ubuntu_common_requirements() {
|
||||
$SUDO apt-get update
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
autoconf \
|
||||
build-essential \
|
||||
ca-certificates \
|
||||
casync \
|
||||
clang \
|
||||
cmake \
|
||||
make \
|
||||
cppcheck \
|
||||
libtool \
|
||||
gcc-arm-none-eabi \
|
||||
bzip2 \
|
||||
liblzma-dev \
|
||||
libarchive-dev \
|
||||
libbz2-dev \
|
||||
capnproto \
|
||||
libcapnp-dev \
|
||||
curl \
|
||||
libcurl4-openssl-dev \
|
||||
git \
|
||||
git-lfs \
|
||||
ffmpeg \
|
||||
libavformat-dev \
|
||||
libavcodec-dev \
|
||||
libavdevice-dev \
|
||||
libavutil-dev \
|
||||
libavfilter-dev \
|
||||
libeigen3-dev \
|
||||
libffi-dev \
|
||||
libglew-dev \
|
||||
libgles2-mesa-dev \
|
||||
libglfw3-dev \
|
||||
libglib2.0-0 \
|
||||
libncurses5-dev \
|
||||
libncursesw5-dev \
|
||||
libomp-dev \
|
||||
libopencv-dev \
|
||||
libpng16-16 \
|
||||
libportaudio2 \
|
||||
libssl-dev \
|
||||
libsqlite3-dev \
|
||||
libusb-1.0-0-dev \
|
||||
libzmq3-dev \
|
||||
libsystemd-dev \
|
||||
locales \
|
||||
opencl-headers \
|
||||
ocl-icd-libopencl1 \
|
||||
ocl-icd-opencl-dev \
|
||||
clinfo \
|
||||
portaudio19-dev \
|
||||
qml-module-qtquick2 \
|
||||
qtmultimedia5-dev \
|
||||
qtlocation5-dev \
|
||||
qtpositioning5-dev \
|
||||
qttools5-dev-tools \
|
||||
libqt5sql5-sqlite \
|
||||
libqt5svg5-dev \
|
||||
libqt5charts5-dev \
|
||||
libqt5serialbus5-dev \
|
||||
libqt5x11extras5-dev \
|
||||
libreadline-dev \
|
||||
libdw1 \
|
||||
valgrind
|
||||
}
|
||||
|
||||
# Install Ubuntu 22.04 LTS packages
|
||||
function install_ubuntu_lts_latest_requirements() {
|
||||
install_ubuntu_common_requirements
|
||||
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
g++-12 \
|
||||
qtbase5-dev \
|
||||
qtchooser \
|
||||
qt5-qmake \
|
||||
qtbase5-dev-tools \
|
||||
python3-dev
|
||||
}
|
||||
|
||||
# Install Ubuntu 20.04 packages
|
||||
function install_ubuntu_focal_requirements() {
|
||||
install_ubuntu_common_requirements
|
||||
|
||||
$SUDO apt-get install -y --no-install-recommends \
|
||||
libavresample-dev \
|
||||
qt5-default \
|
||||
python-dev
|
||||
}
|
||||
|
||||
# Detect OS using /etc/os-release file
|
||||
if [ -f "/etc/os-release" ]; then
|
||||
source /etc/os-release
|
||||
case "$VERSION_CODENAME" in
|
||||
"jammy")
|
||||
install_ubuntu_lts_latest_requirements
|
||||
;;
|
||||
"kinetic")
|
||||
install_ubuntu_lts_latest_requirements
|
||||
;;
|
||||
"focal")
|
||||
install_ubuntu_focal_requirements
|
||||
;;
|
||||
*)
|
||||
echo "$ID $VERSION_ID is unsupported. This setup script is written for Ubuntu 20.04."
|
||||
read -p "Would you like to attempt installation anyway? " -n 1 -r
|
||||
echo ""
|
||||
if [[ ! $REPLY =~ ^[Yy]$ ]]; then
|
||||
exit 1
|
||||
fi
|
||||
if [ "$UBUNTU_CODENAME" = "jammy" ] || [ "$UBUNTU_CODENAME" = "kinetic" ]; then
|
||||
install_ubuntu_lts_latest_requirements
|
||||
else
|
||||
install_ubuntu_focal_requirements
|
||||
fi
|
||||
esac
|
||||
else
|
||||
echo "No /etc/os-release in the system"
|
||||
exit 1
|
||||
fi
|
||||
64
tools/joystick/README.md
Normal file
64
tools/joystick/README.md
Normal file
@@ -0,0 +1,64 @@
|
||||
# Joystick
|
||||
|
||||
**Hardware needed**: device running openpilot, laptop, joystick (optional)
|
||||
|
||||
With joystickd, you can connect your laptop to your comma device over the network and debug controls using a joystick or keyboard.
|
||||
joystickd uses [inputs](https://pypi.org/project/inputs) which supports many common gamepads and joysticks.
|
||||
|
||||
## Usage
|
||||
|
||||
The car must be off, and openpilot must be offroad before starting `joystickd`.
|
||||
|
||||
### Using a keyboard
|
||||
|
||||
SSH into your comma device and start joystickd with the following command:
|
||||
|
||||
```shell
|
||||
tools/joystick/joystickd.py --keyboard
|
||||
```
|
||||
|
||||
The available buttons and axes will print showing their key mappings. In general, the WASD keys control gas and brakes and steering torque in 5% increments.
|
||||
|
||||
### Joystick on your comma three
|
||||
|
||||
Plug the joystick into your comma three aux USB-C port. Then, SSH into the device and start `joystickd.py`.
|
||||
|
||||
### Joystick on your laptop
|
||||
|
||||
In order to use a joystick over the network, we need to run joystickd locally from your laptop and have it send `testJoystick` packets over the network to the comma device.
|
||||
|
||||
1. Connect a joystick to your PC.
|
||||
2. Connect your laptop to your comma device's hotspot and open a new SSH shell. Since joystickd is being run on your laptop, we need to write a parameter to let controlsd know to start in joystick debug mode:
|
||||
```shell
|
||||
# on your comma device
|
||||
echo -n "1" > /data/params/d/JoystickDebugMode
|
||||
```
|
||||
3. Run bridge with your laptop's IP address. This republishes the `testJoystick` packets sent from your laptop so that openpilot can receive them:
|
||||
```shell
|
||||
# on your comma device
|
||||
cereal/messaging/bridge {LAPTOP_IP} testJoystick
|
||||
```
|
||||
4. Start joystickd on your laptop in ZMQ mode.
|
||||
```shell
|
||||
# on your laptop
|
||||
export ZMQ=1
|
||||
tools/joystick/joystickd.py
|
||||
```
|
||||
|
||||
### Web joystick on your mobile device
|
||||
|
||||
A browser-based virtual joystick designed for touch screens. Starts automatically when installed on comma body (non-car robotics platform).
|
||||
For cars, start the web joystick service manually via SSH before starting the car.
|
||||
|
||||
```shell
|
||||
tools/joystick/web.py
|
||||
```
|
||||
|
||||
After starting the car/body, open the web joystick app at this URL: `http://[comma three IP address]:5000`
|
||||
|
||||
---
|
||||
Now start your car and openpilot should go into joystick mode with an alert on startup! The status of the axes will display on the alert, while button statuses print in the shell.
|
||||
|
||||
Make sure the conditions are met in the panda to allow controls (e.g. cruise control engaged). You can also make a modification to the panda code to always allow controls.
|
||||
|
||||
|
||||
119
tools/joystick/joystickd.py
Executable file
119
tools/joystick/joystickd.py
Executable file
@@ -0,0 +1,119 @@
|
||||
#!/usr/bin/env python
|
||||
import os
|
||||
import argparse
|
||||
import threading
|
||||
from inputs import get_gamepad
|
||||
|
||||
import cereal.messaging as messaging
|
||||
from openpilot.common.realtime import Ratekeeper
|
||||
from openpilot.common.numpy_fast import interp, clip
|
||||
from openpilot.common.params import Params
|
||||
from openpilot.tools.lib.kbhit import KBHit
|
||||
|
||||
|
||||
class Keyboard:
|
||||
def __init__(self):
|
||||
self.kb = KBHit()
|
||||
self.axis_increment = 0.05 # 5% of full actuation each key press
|
||||
self.axes_map = {'w': 'gb', 's': 'gb',
|
||||
'a': 'steer', 'd': 'steer'}
|
||||
self.axes_values = {'gb': 0., 'steer': 0.}
|
||||
self.axes_order = ['gb', 'steer']
|
||||
self.cancel = False
|
||||
|
||||
def update(self):
|
||||
key = self.kb.getch().lower()
|
||||
self.cancel = False
|
||||
if key == 'r':
|
||||
self.axes_values = {ax: 0. for ax in self.axes_values}
|
||||
elif key == 'c':
|
||||
self.cancel = True
|
||||
elif key in self.axes_map:
|
||||
axis = self.axes_map[key]
|
||||
incr = self.axis_increment if key in ['w', 'a'] else -self.axis_increment
|
||||
self.axes_values[axis] = clip(self.axes_values[axis] + incr, -1, 1)
|
||||
else:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
class Joystick:
|
||||
def __init__(self, gamepad=False):
|
||||
# TODO: find a way to get this from API, perhaps "inputs" doesn't support it
|
||||
if gamepad:
|
||||
self.cancel_button = 'BTN_NORTH' # (BTN_NORTH=X, ABS_RZ=Right Trigger)
|
||||
accel_axis = 'ABS_Y'
|
||||
steer_axis = 'ABS_RX'
|
||||
else:
|
||||
self.cancel_button = 'BTN_TRIGGER'
|
||||
accel_axis = 'ABS_Y'
|
||||
steer_axis = 'ABS_RZ'
|
||||
self.min_axis_value = {accel_axis: 0., steer_axis: 0.}
|
||||
self.max_axis_value = {accel_axis: 255., steer_axis: 255.}
|
||||
self.axes_values = {accel_axis: 0., steer_axis: 0.}
|
||||
self.axes_order = [accel_axis, steer_axis]
|
||||
self.cancel = False
|
||||
|
||||
def update(self):
|
||||
joystick_event = get_gamepad()[0]
|
||||
event = (joystick_event.code, joystick_event.state)
|
||||
if event[0] == self.cancel_button:
|
||||
if event[1] == 1:
|
||||
self.cancel = True
|
||||
elif event[1] == 0: # state 0 is falling edge
|
||||
self.cancel = False
|
||||
elif event[0] in self.axes_values:
|
||||
self.max_axis_value[event[0]] = max(event[1], self.max_axis_value[event[0]])
|
||||
self.min_axis_value[event[0]] = min(event[1], self.min_axis_value[event[0]])
|
||||
|
||||
norm = -interp(event[1], [self.min_axis_value[event[0]], self.max_axis_value[event[0]]], [-1., 1.])
|
||||
self.axes_values[event[0]] = norm if abs(norm) > 0.05 else 0. # center can be noisy, deadzone of 5%
|
||||
else:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def send_thread(joystick):
|
||||
joystick_sock = messaging.pub_sock('testJoystick')
|
||||
rk = Ratekeeper(100, print_delay_threshold=None)
|
||||
while 1:
|
||||
dat = messaging.new_message('testJoystick')
|
||||
dat.testJoystick.axes = [joystick.axes_values[a] for a in joystick.axes_order]
|
||||
dat.testJoystick.buttons = [joystick.cancel]
|
||||
joystick_sock.send(dat.to_bytes())
|
||||
print('\n' + ', '.join(f'{name}: {round(v, 3)}' for name, v in joystick.axes_values.items()))
|
||||
if "WEB" in os.environ:
|
||||
import requests
|
||||
requests.get("http://"+os.environ["WEB"]+":5000/control/%f/%f" % tuple([joystick.axes_values[a] for a in joystick.axes_order][::-1]), timeout=None)
|
||||
rk.keep_time()
|
||||
|
||||
def joystick_thread(joystick):
|
||||
Params().put_bool('JoystickDebugMode', True)
|
||||
threading.Thread(target=send_thread, args=(joystick,), daemon=True).start()
|
||||
while True:
|
||||
joystick.update()
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Publishes events from your joystick to control your car.\n' +
|
||||
'openpilot must be offroad before starting joysticked.',
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument('--keyboard', action='store_true', help='Use your keyboard instead of a joystick')
|
||||
parser.add_argument('--gamepad', action='store_true', help='Use gamepad configuration instead of joystick')
|
||||
args = parser.parse_args()
|
||||
|
||||
if not Params().get_bool("IsOffroad") and "ZMQ" not in os.environ and "WEB" not in os.environ:
|
||||
print("The car must be off before running joystickd.")
|
||||
exit()
|
||||
|
||||
print()
|
||||
if args.keyboard:
|
||||
print('Gas/brake control: `W` and `S` keys')
|
||||
print('Steering control: `A` and `D` keys')
|
||||
print('Buttons')
|
||||
print('- `R`: Resets axes')
|
||||
print('- `C`: Cancel cruise control')
|
||||
else:
|
||||
print('Using joystick, make sure to run cereal/messaging/bridge on your device if running over the network!')
|
||||
|
||||
joystick = Keyboard() if args.keyboard else Joystick(args.gamepad)
|
||||
joystick_thread(joystick)
|
||||
BIN
tools/joystick/steer.gif
Normal file
BIN
tools/joystick/steer.gif
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 7.0 MiB |
92
tools/latencylogger/README.md
Normal file
92
tools/latencylogger/README.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# LatencyLogger
|
||||
|
||||
LatencyLogger is a tool to track the time from first pixel to actuation. Timestamps are printed in a table as well as plotted in a graph. Start openpilot with `LOG_TIMESTAMPS=1` set to enable the necessary logging.
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
$ python latency_logger.py -h
|
||||
usage: latency_logger.py [-h] [--relative] [--demo] [--plot] [route_or_segment_name]
|
||||
|
||||
A tool for analyzing openpilot's end-to-end latency
|
||||
|
||||
positional arguments:
|
||||
route_or_segment_name
|
||||
The route to print (default: None)
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
--relative Make timestamps relative to the start of each frame (default: False)
|
||||
--demo Use the demo route instead of providing one (default: False)
|
||||
--plot If a plot should be generated (default: False)
|
||||
--offset Offset service to better visualize overlap (default: False)
|
||||
```
|
||||
To timestamp an event, use `LOGT("msg")` in c++ code or `cloudlog.timestamp("msg")` in python code. If the print is warning for frameId assignment ambiguity, use `LOGT(frameId ,"msg")`.
|
||||
|
||||
## Examples
|
||||
|
||||
Timestamps are visualized as diamonds
|
||||
|
||||
| | Relative | Absolute |
|
||||
| ------------- | ------------- | ------------- |
|
||||
| Inline |  |  |
|
||||
| Offset |  |  |
|
||||
|
||||
Printed timestamps of a frame with internal durations.
|
||||
```
|
||||
Frame ID: 1202
|
||||
camerad
|
||||
wideRoadCameraState start of frame 0.0
|
||||
roadCameraState start of frame 0.049583
|
||||
wideRoadCameraState published 35.01206
|
||||
WideRoadCamera: Image set 35.020028
|
||||
roadCameraState published 38.508261
|
||||
RoadCamera: Image set 38.520344
|
||||
RoadCamera: Transformed 38.616176
|
||||
wideRoadCameraState.processingTime 3.152403049170971
|
||||
roadCameraState.processingTime 6.453451234847307
|
||||
modeld
|
||||
Image added 40.909841
|
||||
Extra image added 42.515027
|
||||
Execution finished 63.002552
|
||||
modelV2 published 63.148747
|
||||
modelV2.modelExecutionTime 23.62649142742157
|
||||
modelV2.gpuExecutionTime 0.0
|
||||
plannerd
|
||||
lateralPlan published 66.915049
|
||||
longitudinalPlan published 69.715999
|
||||
lateralPlan.solverExecutionTime 0.8170719956979156
|
||||
longitudinalPlan.solverExecutionTime 0.5619999719783664
|
||||
controlsd
|
||||
Data sampled 70.217763
|
||||
Events updated 71.037178
|
||||
sendcan published 72.278775
|
||||
controlsState published 72.825226
|
||||
Data sampled 80.008354
|
||||
Events updated 80.787666
|
||||
sendcan published 81.849682
|
||||
controlsState published 82.238323
|
||||
Data sampled 90.521123
|
||||
Events updated 91.626003
|
||||
sendcan published 93.413218
|
||||
controlsState published 94.143989
|
||||
Data sampled 100.991497
|
||||
Events updated 101.973774
|
||||
sendcan published 103.565575
|
||||
controlsState published 104.146088
|
||||
Data sampled 110.284387
|
||||
Events updated 111.183541
|
||||
sendcan published 112.981692
|
||||
controlsState published 113.731994
|
||||
boardd
|
||||
sending sendcan to panda: 250027001751393037323631 81.928119
|
||||
sendcan sent to panda: 250027001751393037323631 82.164834
|
||||
sending sendcan to panda: 250027001751393037323631 93.569986
|
||||
sendcan sent to panda: 250027001751393037323631 93.92795
|
||||
sending sendcan to panda: 250027001751393037323631 103.689167
|
||||
sendcan sent to panda: 250027001751393037323631 104.012235
|
||||
sending sendcan to panda: 250027001751393037323631 113.109555
|
||||
sendcan sent to panda: 250027001751393037323631 113.525487
|
||||
sending sendcan to panda: 250027001751393037323631 122.508434
|
||||
sendcan sent to panda: 250027001751393037323631 122.834314
|
||||
```
|
||||
244
tools/latencylogger/latency_logger.py
Executable file
244
tools/latencylogger/latency_logger.py
Executable file
@@ -0,0 +1,244 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import json
|
||||
import matplotlib.patches as mpatches
|
||||
import matplotlib.pyplot as plt
|
||||
import mpld3
|
||||
import sys
|
||||
from bisect import bisect_left, bisect_right
|
||||
from collections import defaultdict
|
||||
|
||||
from openpilot.tools.lib.logreader import logreader_from_route_or_segment
|
||||
|
||||
DEMO_ROUTE = "9f583b1d93915c31|2022-05-18--10-49-51--0"
|
||||
|
||||
SERVICES = ['camerad', 'modeld', 'plannerd', 'controlsd', 'boardd']
|
||||
# Retrieve controlsd frameId from lateralPlan, mismatch with longitudinalPlan will be ignored
|
||||
MONOTIME_KEYS = ['modelMonoTime', 'lateralPlanMonoTime']
|
||||
MSGQ_TO_SERVICE = {
|
||||
'roadCameraState': 'camerad',
|
||||
'wideRoadCameraState': 'camerad',
|
||||
'modelV2': 'modeld',
|
||||
'lateralPlan': 'plannerd',
|
||||
'longitudinalPlan': 'plannerd',
|
||||
'sendcan': 'controlsd',
|
||||
'controlsState': 'controlsd'
|
||||
}
|
||||
SERVICE_TO_DURATIONS = {
|
||||
'camerad': ['processingTime'],
|
||||
'modeld': ['modelExecutionTime', 'gpuExecutionTime'],
|
||||
'plannerd': ['solverExecutionTime'],
|
||||
}
|
||||
|
||||
def read_logs(lr):
|
||||
data = defaultdict(lambda: defaultdict(lambda: defaultdict(list)))
|
||||
mono_to_frame = {}
|
||||
frame_mismatches = []
|
||||
frame_id_fails = 0
|
||||
latest_sendcan_monotime = 0
|
||||
for msg in lr:
|
||||
if msg.which() == 'sendcan':
|
||||
latest_sendcan_monotime = msg.logMonoTime
|
||||
continue
|
||||
|
||||
if msg.which() in MSGQ_TO_SERVICE:
|
||||
service = MSGQ_TO_SERVICE[msg.which()]
|
||||
msg_obj = getattr(msg, msg.which())
|
||||
|
||||
frame_id = -1
|
||||
if hasattr(msg_obj, "frameId"):
|
||||
frame_id = msg_obj.frameId
|
||||
else:
|
||||
continue_outer = False
|
||||
for key in MONOTIME_KEYS:
|
||||
if hasattr(msg_obj, key):
|
||||
if getattr(msg_obj, key) == 0:
|
||||
# Filter out controlsd messages which arrive before the camera loop
|
||||
continue_outer = True
|
||||
elif getattr(msg_obj, key) in mono_to_frame:
|
||||
frame_id = mono_to_frame[getattr(msg_obj, key)]
|
||||
if continue_outer:
|
||||
continue
|
||||
if frame_id == -1:
|
||||
frame_id_fails += 1
|
||||
continue
|
||||
mono_to_frame[msg.logMonoTime] = frame_id
|
||||
data['timestamp'][frame_id][service].append((msg.which()+" published", msg.logMonoTime))
|
||||
|
||||
next_service = SERVICES[SERVICES.index(service)+1]
|
||||
if not data['start'][frame_id][next_service]:
|
||||
data['start'][frame_id][next_service] = msg.logMonoTime
|
||||
data['end'][frame_id][service] = msg.logMonoTime
|
||||
|
||||
if service in SERVICE_TO_DURATIONS:
|
||||
for duration in SERVICE_TO_DURATIONS[service]:
|
||||
data['duration'][frame_id][service].append((msg.which()+"."+duration, getattr(msg_obj, duration)))
|
||||
|
||||
if service == SERVICES[0]:
|
||||
data['timestamp'][frame_id][service].append((msg.which()+" start of frame", msg_obj.timestampSof))
|
||||
if not data['start'][frame_id][service]:
|
||||
data['start'][frame_id][service] = msg_obj.timestampSof
|
||||
elif msg.which() == 'controlsState':
|
||||
# Sendcan is published before controlsState, but the frameId is retrieved in CS
|
||||
data['timestamp'][frame_id][service].append(("sendcan published", latest_sendcan_monotime))
|
||||
elif msg.which() == 'modelV2':
|
||||
if msg_obj.frameIdExtra != frame_id:
|
||||
frame_mismatches.append(frame_id)
|
||||
|
||||
if frame_id_fails > 20:
|
||||
print("Warning, many frameId fetch fails", frame_id_fails)
|
||||
if len(frame_mismatches) > 20:
|
||||
print("Warning, many frame mismatches", len(frame_mismatches))
|
||||
return (data, frame_mismatches)
|
||||
|
||||
# This is not needed in 3.10 as a "key" parameter is added to bisect
|
||||
class KeyifyList(object):
|
||||
def __init__(self, inner, key):
|
||||
self.inner = inner
|
||||
self.key = key
|
||||
def __len__(self):
|
||||
return len(self.inner)
|
||||
def __getitem__(self, k):
|
||||
return self.key(self.inner[k])
|
||||
|
||||
def find_frame_id(time, service, start_times, end_times):
|
||||
left = bisect_left(KeyifyList(list(start_times.items()),
|
||||
lambda x: x[1][service] if x[1][service] else -1), time) - 1
|
||||
right = bisect_right(KeyifyList(list(end_times.items()),
|
||||
lambda x: x[1][service] if x[1][service] else float("inf")), time)
|
||||
return left, right
|
||||
|
||||
def find_t0(start_times, frame_id=-1):
|
||||
frame_id = frame_id if frame_id > -1 else min(start_times.keys())
|
||||
m = max(start_times.keys())
|
||||
while frame_id <= m:
|
||||
for service in SERVICES:
|
||||
if start_times[frame_id][service]:
|
||||
return start_times[frame_id][service]
|
||||
frame_id += 1
|
||||
raise Exception('No start time has been set')
|
||||
|
||||
def insert_cloudlogs(lr, timestamps, start_times, end_times):
|
||||
# at least one cloudlog must be made in controlsd
|
||||
|
||||
t0 = find_t0(start_times)
|
||||
failed_inserts = 0
|
||||
latest_controls_frameid = 0
|
||||
for msg in lr:
|
||||
if msg.which() == "logMessage":
|
||||
jmsg = json.loads(msg.logMessage)
|
||||
if "timestamp" in jmsg['msg']:
|
||||
time = int(jmsg['msg']['timestamp']['time'])
|
||||
service = jmsg['ctx']['daemon']
|
||||
event = jmsg['msg']['timestamp']['event']
|
||||
if time < t0:
|
||||
# Filter out controlsd messages which arrive before the camera loop
|
||||
continue
|
||||
|
||||
if "frame_id" in jmsg['msg']['timestamp']:
|
||||
timestamps[int(jmsg['msg']['timestamp']['frame_id'])][service].append((event, time))
|
||||
continue
|
||||
|
||||
if service == "boardd":
|
||||
timestamps[latest_controls_frameid][service].append((event, time))
|
||||
end_times[latest_controls_frameid][service] = time
|
||||
else:
|
||||
frame_id = find_frame_id(time, service, start_times, end_times)
|
||||
if frame_id:
|
||||
if frame_id[0] != frame_id[1]:
|
||||
event += " (warning: ambiguity)"
|
||||
frame_id = frame_id[0]
|
||||
if service == 'controlsd':
|
||||
latest_controls_frameid = frame_id
|
||||
timestamps[frame_id][service].append((event, time))
|
||||
else:
|
||||
failed_inserts += 1
|
||||
|
||||
if latest_controls_frameid == 0:
|
||||
print("Warning: failed to bind boardd logs to a frame ID. Add a timestamp cloudlog in controlsd.")
|
||||
elif failed_inserts > len(timestamps):
|
||||
print(f"Warning: failed to bind {failed_inserts} cloudlog timestamps to a frame ID")
|
||||
|
||||
def print_timestamps(timestamps, durations, start_times, relative):
|
||||
t0 = find_t0(start_times)
|
||||
for frame_id in timestamps.keys():
|
||||
print('='*80)
|
||||
print("Frame ID:", frame_id)
|
||||
if relative:
|
||||
t0 = find_t0(start_times, frame_id)
|
||||
|
||||
for service in SERVICES:
|
||||
print(" "+service)
|
||||
events = timestamps[frame_id][service]
|
||||
for event, time in sorted(events, key = lambda x: x[1]):
|
||||
print(" "+'%-53s%-53s' %(event, str((time-t0)/1e6)))
|
||||
for event, time in durations[frame_id][service]:
|
||||
print(" "+'%-53s%-53s' %(event, str(time*1000)))
|
||||
|
||||
def graph_timestamps(timestamps, start_times, end_times, relative, offset_services=False, title=""):
|
||||
# mpld3 doesn't convert properly to D3 font sizes
|
||||
plt.rcParams.update({'font.size': 18})
|
||||
|
||||
t0 = find_t0(start_times)
|
||||
fig, ax = plt.subplots()
|
||||
ax.set_xlim(0, 130 if relative else 750)
|
||||
ax.set_ylim(0, 17)
|
||||
ax.set_xlabel('Time (milliseconds)')
|
||||
colors = ['blue', 'green', 'red', 'yellow', 'purple']
|
||||
offsets = [[0, -5*j] for j in range(len(SERVICES))] if offset_services else None
|
||||
height = 0.3 if offset_services else 0.9
|
||||
assert len(colors) == len(SERVICES), 'Each service needs a color'
|
||||
|
||||
points = {"x": [], "y": [], "labels": []}
|
||||
for i, (frame_id, services) in enumerate(timestamps.items()):
|
||||
if relative:
|
||||
t0 = find_t0(start_times, frame_id)
|
||||
service_bars = []
|
||||
for service, events in services.items():
|
||||
if start_times[frame_id][service] and end_times[frame_id][service]:
|
||||
start = start_times[frame_id][service]
|
||||
end = end_times[frame_id][service]
|
||||
service_bars.append(((start-t0)/1e6,(end-start)/1e6))
|
||||
for event in events:
|
||||
points['x'].append((event[1]-t0)/1e6)
|
||||
points['y'].append(i)
|
||||
points['labels'].append(event[0])
|
||||
ax.broken_barh(service_bars, (i-height/2, height), facecolors=(colors), alpha=0.5, offsets=offsets)
|
||||
|
||||
scatter = ax.scatter(points['x'], points['y'], marker='d', edgecolor='black')
|
||||
tooltip = mpld3.plugins.PointLabelTooltip(scatter, labels=points['labels'])
|
||||
mpld3.plugins.connect(fig, tooltip)
|
||||
|
||||
plt.title(title)
|
||||
# Set size relative window size is not trivial: https://github.com/mpld3/mpld3/issues/65
|
||||
fig.set_size_inches(18, 9)
|
||||
plt.legend(handles=[mpatches.Patch(color=colors[i], label=SERVICES[i]) for i in range(len(SERVICES))])
|
||||
return fig
|
||||
|
||||
def get_timestamps(lr):
|
||||
lr = list(lr)
|
||||
data, frame_mismatches = read_logs(lr)
|
||||
insert_cloudlogs(lr, data['timestamp'], data['start'], data['end'])
|
||||
return data, frame_mismatches
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="A tool for analyzing openpilot's end-to-end latency",
|
||||
formatter_class = argparse.ArgumentDefaultsHelpFormatter)
|
||||
parser.add_argument("--relative", action="store_true", help="Make timestamps relative to the start of each frame")
|
||||
parser.add_argument("--demo", action="store_true", help="Use the demo route instead of providing one")
|
||||
parser.add_argument("--plot", action="store_true", help="If a plot should be generated")
|
||||
parser.add_argument("--offset", action="store_true", help="Vertically offset service to better visualize overlap")
|
||||
parser.add_argument("route_or_segment_name", nargs='?', help="The route to print")
|
||||
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
args = parser.parse_args()
|
||||
|
||||
r = DEMO_ROUTE if args.demo else args.route_or_segment_name.strip()
|
||||
lr = logreader_from_route_or_segment(r, sort_by_time=True)
|
||||
|
||||
data, _ = get_timestamps(lr)
|
||||
print_timestamps(data['timestamp'], data['duration'], data['start'], args.relative)
|
||||
if args.plot:
|
||||
mpld3.show(graph_timestamps(data['timestamp'], data['start'], data['end'], args.relative, offset_services=args.offset, title=r))
|
||||
51
tools/lib/README.md
Normal file
51
tools/lib/README.md
Normal file
@@ -0,0 +1,51 @@
|
||||
## LogReader
|
||||
|
||||
Route is a class for conveniently accessing all the [logs](/system/loggerd/) from your routes. The LogReader class reads the non-video logs, i.e. rlog.bz2 and qlog.bz2. There's also a matching FrameReader class for reading the videos.
|
||||
|
||||
```python
|
||||
from openpilot.tools.lib.route import Route
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
r = Route("a2a0ccea32023010|2023-07-27--13-01-19")
|
||||
|
||||
# get a list of paths for the route's rlog files
|
||||
print(r.log_paths())
|
||||
|
||||
# and road camera (fcamera.hevc) files
|
||||
print(r.camera_paths())
|
||||
|
||||
# setup a LogReader to read the route's first rlog
|
||||
lr = LogReader(r.log_paths()[0])
|
||||
|
||||
# print out all the messages in the log
|
||||
import codecs
|
||||
codecs.register_error("strict", codecs.backslashreplace_errors)
|
||||
for msg in lr:
|
||||
print(msg)
|
||||
|
||||
# setup a LogReader for the route's second qlog
|
||||
lr = LogReader(r.log_paths()[1])
|
||||
|
||||
# print all the steering angles values from the log
|
||||
for msg in lr:
|
||||
if msg.which() == "carState":
|
||||
print(msg.carState.steeringAngleDeg)
|
||||
```
|
||||
|
||||
### MultiLogIterator
|
||||
|
||||
`MultiLogIterator` is similar to `LogReader`, but reads multiple logs.
|
||||
|
||||
```python
|
||||
from openpilot.tools.lib.route import Route
|
||||
from openpilot.tools.lib.logreader import MultiLogIterator
|
||||
|
||||
# setup a MultiLogIterator to read all the logs in the route
|
||||
r = Route("a2a0ccea32023010|2023-07-27--13-01-19")
|
||||
lr = MultiLogIterator(r.log_paths())
|
||||
|
||||
# print all the steering angles values from all the logs in the route
|
||||
for msg in lr:
|
||||
if msg.which() == "carState":
|
||||
print(msg.carState.steeringAngleDeg)
|
||||
```
|
||||
0
tools/lib/__init__.py
Normal file
0
tools/lib/__init__.py
Normal file
34
tools/lib/api.py
Normal file
34
tools/lib/api.py
Normal file
@@ -0,0 +1,34 @@
|
||||
import os
|
||||
import requests
|
||||
API_HOST = os.getenv('API_HOST', 'https://api.commadotai.com')
|
||||
|
||||
class CommaApi():
|
||||
def __init__(self, token=None):
|
||||
self.session = requests.Session()
|
||||
self.session.headers['User-agent'] = 'OpenpilotTools'
|
||||
if token:
|
||||
self.session.headers['Authorization'] = 'JWT ' + token
|
||||
|
||||
def request(self, method, endpoint, **kwargs):
|
||||
resp = self.session.request(method, API_HOST + '/' + endpoint, **kwargs)
|
||||
resp_json = resp.json()
|
||||
if isinstance(resp_json, dict) and resp_json.get('error'):
|
||||
if resp.status_code in [401, 403]:
|
||||
raise UnauthorizedError('Unauthorized. Authenticate with tools/lib/auth.py')
|
||||
|
||||
e = APIError(str(resp.status_code) + ":" + resp_json.get('description', str(resp_json['error'])))
|
||||
e.status_code = resp.status_code
|
||||
raise e
|
||||
return resp_json
|
||||
|
||||
def get(self, endpoint, **kwargs):
|
||||
return self.request('GET', endpoint, **kwargs)
|
||||
|
||||
def post(self, endpoint, **kwargs):
|
||||
return self.request('POST', endpoint, **kwargs)
|
||||
|
||||
class APIError(Exception):
|
||||
pass
|
||||
|
||||
class UnauthorizedError(Exception):
|
||||
pass
|
||||
145
tools/lib/auth.py
Executable file
145
tools/lib/auth.py
Executable file
@@ -0,0 +1,145 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
Usage::
|
||||
|
||||
usage: auth.py [-h] [{google,apple,github,jwt}] [jwt]
|
||||
|
||||
Login to your comma account
|
||||
|
||||
positional arguments:
|
||||
{google,apple,github,jwt}
|
||||
jwt
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
|
||||
|
||||
Examples::
|
||||
|
||||
./auth.py # Log in with google account
|
||||
./auth.py github # Log in with GitHub Account
|
||||
./auth.py jwt ey......hw # Log in with a JWT from https://jwt.comma.ai, for use in CI
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import pprint
|
||||
import webbrowser
|
||||
from http.server import BaseHTTPRequestHandler, HTTPServer
|
||||
from typing import Any, Dict
|
||||
from urllib.parse import parse_qs, urlencode
|
||||
|
||||
from openpilot.tools.lib.api import APIError, CommaApi, UnauthorizedError
|
||||
from openpilot.tools.lib.auth_config import set_token, get_token
|
||||
|
||||
PORT = 3000
|
||||
|
||||
|
||||
class ClientRedirectServer(HTTPServer):
|
||||
query_params: Dict[str, Any] = {}
|
||||
|
||||
|
||||
class ClientRedirectHandler(BaseHTTPRequestHandler):
|
||||
def do_GET(self):
|
||||
if not self.path.startswith('/auth'):
|
||||
self.send_response(204)
|
||||
return
|
||||
|
||||
query = self.path.split('?', 1)[-1]
|
||||
query_parsed = parse_qs(query, keep_blank_values=True)
|
||||
self.server.query_params = query_parsed
|
||||
|
||||
self.send_response(200)
|
||||
self.send_header('Content-type', 'text/plain')
|
||||
self.end_headers()
|
||||
self.wfile.write(b'Return to the CLI to continue')
|
||||
|
||||
def log_message(self, *args): # pylint: disable=redefined-builtin
|
||||
pass # this prevent http server from dumping messages to stdout
|
||||
|
||||
|
||||
def auth_redirect_link(method):
|
||||
provider_id = {
|
||||
'google': 'g',
|
||||
'apple': 'a',
|
||||
'github': 'h',
|
||||
}[method]
|
||||
|
||||
params = {
|
||||
'redirect_uri': f"https://api.comma.ai/v2/auth/{provider_id}/redirect/",
|
||||
'state': f'service,localhost:{PORT}',
|
||||
}
|
||||
|
||||
if method == 'google':
|
||||
params.update({
|
||||
'type': 'web_server',
|
||||
'client_id': '45471411055-ornt4svd2miog6dnopve7qtmh5mnu6id.apps.googleusercontent.com',
|
||||
'response_type': 'code',
|
||||
'scope': 'https://www.googleapis.com/auth/userinfo.email',
|
||||
'prompt': 'select_account',
|
||||
})
|
||||
return 'https://accounts.google.com/o/oauth2/auth?' + urlencode(params)
|
||||
elif method == 'github':
|
||||
params.update({
|
||||
'client_id': '28c4ecb54bb7272cb5a4',
|
||||
'scope': 'read:user',
|
||||
})
|
||||
return 'https://github.com/login/oauth/authorize?' + urlencode(params)
|
||||
elif method == 'apple':
|
||||
params.update({
|
||||
'client_id': 'ai.comma.login',
|
||||
'response_type': 'code',
|
||||
'response_mode': 'form_post',
|
||||
'scope': 'name email',
|
||||
})
|
||||
return 'https://appleid.apple.com/auth/authorize?' + urlencode(params)
|
||||
else:
|
||||
raise NotImplementedError(f"no redirect implemented for method {method}")
|
||||
|
||||
|
||||
def login(method):
|
||||
oauth_uri = auth_redirect_link(method)
|
||||
|
||||
web_server = ClientRedirectServer(('localhost', PORT), ClientRedirectHandler)
|
||||
print(f'To sign in, use your browser and navigate to {oauth_uri}')
|
||||
webbrowser.open(oauth_uri, new=2)
|
||||
|
||||
while True:
|
||||
web_server.handle_request()
|
||||
if 'code' in web_server.query_params:
|
||||
break
|
||||
elif 'error' in web_server.query_params:
|
||||
print('Authentication Error: "{}". Description: "{}" '.format(
|
||||
web_server.query_params['error'],
|
||||
web_server.query_params.get('error_description')), file=sys.stderr)
|
||||
break
|
||||
|
||||
try:
|
||||
auth_resp = CommaApi().post('v2/auth/', data={'code': web_server.query_params['code'], 'provider': web_server.query_params['provider']})
|
||||
set_token(auth_resp['access_token'])
|
||||
except APIError as e:
|
||||
print(f'Authentication Error: {e}', file=sys.stderr)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
parser = argparse.ArgumentParser(description='Login to your comma account')
|
||||
parser.add_argument('method', default='google', const='google', nargs='?', choices=['google', 'apple', 'github', 'jwt'])
|
||||
parser.add_argument('jwt', nargs='?')
|
||||
|
||||
args = parser.parse_args()
|
||||
if args.method == 'jwt':
|
||||
if args.jwt is None:
|
||||
print("method JWT selected, but no JWT was provided")
|
||||
exit(1)
|
||||
|
||||
set_token(args.jwt)
|
||||
else:
|
||||
login(args.method)
|
||||
|
||||
try:
|
||||
me = CommaApi(token=get_token()).get('/v1/me')
|
||||
print("Authenticated!")
|
||||
pprint.pprint(me)
|
||||
except UnauthorizedError:
|
||||
print("Got invalid JWT")
|
||||
exit(1)
|
||||
34
tools/lib/auth_config.py
Normal file
34
tools/lib/auth_config.py
Normal file
@@ -0,0 +1,34 @@
|
||||
import json
|
||||
import os
|
||||
from openpilot.common.file_helpers import mkdirs_exists_ok
|
||||
from openpilot.system.hardware import PC
|
||||
|
||||
|
||||
class MissingAuthConfigError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
if PC:
|
||||
CONFIG_DIR = os.path.expanduser('~/.comma')
|
||||
else:
|
||||
CONFIG_DIR = "/tmp/.comma"
|
||||
|
||||
mkdirs_exists_ok(CONFIG_DIR)
|
||||
|
||||
|
||||
def get_token():
|
||||
try:
|
||||
with open(os.path.join(CONFIG_DIR, 'auth.json')) as f:
|
||||
auth = json.load(f)
|
||||
return auth['access_token']
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def set_token(token):
|
||||
with open(os.path.join(CONFIG_DIR, 'auth.json'), 'w') as f:
|
||||
json.dump({'access_token': token}, f)
|
||||
|
||||
|
||||
def clear_token():
|
||||
os.unlink(os.path.join(CONFIG_DIR, 'auth.json'))
|
||||
63
tools/lib/bootlog.py
Normal file
63
tools/lib/bootlog.py
Normal file
@@ -0,0 +1,63 @@
|
||||
import datetime
|
||||
import functools
|
||||
import re
|
||||
from typing import List, Optional
|
||||
|
||||
from openpilot.tools.lib.auth_config import get_token
|
||||
from openpilot.tools.lib.api import CommaApi
|
||||
from openpilot.tools.lib.helpers import RE, timestamp_to_datetime
|
||||
|
||||
|
||||
@functools.total_ordering
|
||||
class Bootlog:
|
||||
def __init__(self, url: str):
|
||||
self._url = url
|
||||
|
||||
r = re.search(RE.BOOTLOG_NAME, url)
|
||||
if not r:
|
||||
raise Exception(f"Unable to parse: {url}")
|
||||
|
||||
self._dongle_id = r.group('dongle_id')
|
||||
self._timestamp = r.group('timestamp')
|
||||
|
||||
@property
|
||||
def url(self) -> str:
|
||||
return self._url
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str:
|
||||
return self._dongle_id
|
||||
|
||||
@property
|
||||
def timestamp(self) -> str:
|
||||
return self._timestamp
|
||||
|
||||
@property
|
||||
def datetime(self) -> datetime.datetime:
|
||||
return timestamp_to_datetime(self._timestamp)
|
||||
|
||||
def __str__(self):
|
||||
return f"{self._dongle_id}|{self._timestamp}"
|
||||
|
||||
def __eq__(self, b) -> bool:
|
||||
if not isinstance(b, Bootlog):
|
||||
return False
|
||||
return self.datetime == b.datetime
|
||||
|
||||
def __lt__(self, b) -> bool:
|
||||
if not isinstance(b, Bootlog):
|
||||
return False
|
||||
return self.datetime < b.datetime
|
||||
|
||||
def get_bootlog_from_id(bootlog_id: str) -> Optional[Bootlog]:
|
||||
# TODO: implement an API endpoint for this
|
||||
bl = Bootlog(bootlog_id)
|
||||
for b in get_bootlogs(bl.dongle_id):
|
||||
if b == bl:
|
||||
return b
|
||||
return None
|
||||
|
||||
def get_bootlogs(dongle_id: str) -> List[Bootlog]:
|
||||
api = CommaApi(get_token())
|
||||
r = api.get(f'v1/devices/{dongle_id}/bootlogs')
|
||||
return [Bootlog(b) for b in r]
|
||||
15
tools/lib/cache.py
Normal file
15
tools/lib/cache.py
Normal file
@@ -0,0 +1,15 @@
|
||||
import os
|
||||
import urllib.parse
|
||||
from openpilot.common.file_helpers import mkdirs_exists_ok
|
||||
|
||||
DEFAULT_CACHE_DIR = os.path.expanduser("~/.commacache")
|
||||
|
||||
def cache_path_for_file_path(fn, cache_prefix=None):
|
||||
dir_ = os.path.join(DEFAULT_CACHE_DIR, "local")
|
||||
mkdirs_exists_ok(dir_)
|
||||
fn_parsed = urllib.parse.urlparse(fn)
|
||||
if fn_parsed.scheme == '':
|
||||
cache_fn = os.path.abspath(fn).replace("/", "_")
|
||||
else:
|
||||
cache_fn = f'{fn_parsed.hostname}_{fn_parsed.path.replace("/", "_")}'
|
||||
return os.path.join(dir_, cache_fn)
|
||||
2
tools/lib/exceptions.py
Normal file
2
tools/lib/exceptions.py
Normal file
@@ -0,0 +1,2 @@
|
||||
class DataUnreadableError(Exception):
|
||||
pass
|
||||
11
tools/lib/filereader.py
Normal file
11
tools/lib/filereader.py
Normal file
@@ -0,0 +1,11 @@
|
||||
import os
|
||||
from openpilot.tools.lib.url_file import URLFile
|
||||
|
||||
DATA_ENDPOINT = os.getenv("DATA_ENDPOINT", "http://data-raw.comma.internal/")
|
||||
|
||||
def FileReader(fn, debug=False):
|
||||
if fn.startswith("cd:/"):
|
||||
fn = fn.replace("cd:/", DATA_ENDPOINT)
|
||||
if fn.startswith(("http://", "https://")):
|
||||
return URLFile(fn, debug=debug)
|
||||
return open(fn, "rb")
|
||||
609
tools/lib/framereader.py
Normal file
609
tools/lib/framereader.py
Normal file
@@ -0,0 +1,609 @@
|
||||
# pylint: skip-file
|
||||
import json
|
||||
import os
|
||||
import pickle
|
||||
import struct
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
from enum import IntEnum
|
||||
from functools import wraps
|
||||
|
||||
import numpy as np
|
||||
from lru import LRU
|
||||
|
||||
import _io
|
||||
from openpilot.tools.lib.cache import cache_path_for_file_path
|
||||
from openpilot.tools.lib.exceptions import DataUnreadableError
|
||||
from openpilot.common.file_helpers import atomic_write_in_dir
|
||||
|
||||
from openpilot.tools.lib.filereader import FileReader
|
||||
|
||||
HEVC_SLICE_B = 0
|
||||
HEVC_SLICE_P = 1
|
||||
HEVC_SLICE_I = 2
|
||||
|
||||
|
||||
class GOPReader:
|
||||
def get_gop(self, num):
|
||||
# returns (start_frame_num, num_frames, frames_to_skip, gop_data)
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
class DoNothingContextManager:
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *x):
|
||||
pass
|
||||
|
||||
|
||||
class FrameType(IntEnum):
|
||||
raw = 1
|
||||
h265_stream = 2
|
||||
|
||||
|
||||
def fingerprint_video(fn):
|
||||
with FileReader(fn) as f:
|
||||
header = f.read(4)
|
||||
if len(header) == 0:
|
||||
raise DataUnreadableError(f"{fn} is empty")
|
||||
elif header == b"\x00\xc0\x12\x00":
|
||||
return FrameType.raw
|
||||
elif header == b"\x00\x00\x00\x01":
|
||||
if 'hevc' in fn:
|
||||
return FrameType.h265_stream
|
||||
else:
|
||||
raise NotImplementedError(fn)
|
||||
else:
|
||||
raise NotImplementedError(fn)
|
||||
|
||||
|
||||
def ffprobe(fn, fmt=None):
|
||||
cmd = ["ffprobe",
|
||||
"-v", "quiet",
|
||||
"-print_format", "json",
|
||||
"-show_format", "-show_streams"]
|
||||
if fmt:
|
||||
cmd += ["-f", fmt]
|
||||
cmd += [fn]
|
||||
|
||||
try:
|
||||
ffprobe_output = subprocess.check_output(cmd)
|
||||
except subprocess.CalledProcessError as e:
|
||||
raise DataUnreadableError(fn) from e
|
||||
|
||||
return json.loads(ffprobe_output)
|
||||
|
||||
|
||||
def vidindex(fn, typ):
|
||||
vidindex_dir = os.path.join(os.path.dirname(os.path.realpath(__file__)), "vidindex")
|
||||
vidindex = os.path.join(vidindex_dir, "vidindex")
|
||||
|
||||
subprocess.check_call(["make"], cwd=vidindex_dir, stdout=subprocess.DEVNULL)
|
||||
|
||||
with tempfile.NamedTemporaryFile() as prefix_f, \
|
||||
tempfile.NamedTemporaryFile() as index_f:
|
||||
try:
|
||||
subprocess.check_call([vidindex, typ, fn, prefix_f.name, index_f.name])
|
||||
except subprocess.CalledProcessError as e:
|
||||
raise DataUnreadableError(f"vidindex failed on file {fn}") from e
|
||||
with open(index_f.name, "rb") as f:
|
||||
index = f.read()
|
||||
with open(prefix_f.name, "rb") as f:
|
||||
prefix = f.read()
|
||||
|
||||
index = np.frombuffer(index, np.uint32).reshape(-1, 2)
|
||||
|
||||
assert index[-1, 0] == 0xFFFFFFFF
|
||||
assert index[-1, 1] == os.path.getsize(fn)
|
||||
|
||||
return index, prefix
|
||||
|
||||
|
||||
def cache_fn(func):
|
||||
@wraps(func)
|
||||
def cache_inner(fn, *args, **kwargs):
|
||||
if kwargs.pop('no_cache', None):
|
||||
cache_path = None
|
||||
else:
|
||||
cache_prefix = kwargs.pop('cache_prefix', None)
|
||||
cache_path = cache_path_for_file_path(fn, cache_prefix)
|
||||
|
||||
if cache_path and os.path.exists(cache_path):
|
||||
with open(cache_path, "rb") as cache_file:
|
||||
cache_value = pickle.load(cache_file)
|
||||
else:
|
||||
cache_value = func(fn, *args, **kwargs)
|
||||
|
||||
if cache_path:
|
||||
with atomic_write_in_dir(cache_path, mode="wb", overwrite=True) as cache_file:
|
||||
pickle.dump(cache_value, cache_file, -1)
|
||||
|
||||
return cache_value
|
||||
|
||||
return cache_inner
|
||||
|
||||
|
||||
@cache_fn
|
||||
def index_stream(fn, typ):
|
||||
assert typ in ("hevc", )
|
||||
|
||||
with FileReader(fn) as f:
|
||||
assert os.path.exists(f.name), fn
|
||||
index, prefix = vidindex(f.name, typ)
|
||||
probe = ffprobe(f.name, typ)
|
||||
|
||||
return {
|
||||
'index': index,
|
||||
'global_prefix': prefix,
|
||||
'probe': probe
|
||||
}
|
||||
|
||||
|
||||
def index_videos(camera_paths, cache_prefix=None):
|
||||
"""Requires that paths in camera_paths are contiguous and of the same type."""
|
||||
if len(camera_paths) < 1:
|
||||
raise ValueError("must provide at least one video to index")
|
||||
|
||||
frame_type = fingerprint_video(camera_paths[0])
|
||||
for fn in camera_paths:
|
||||
index_video(fn, frame_type, cache_prefix)
|
||||
|
||||
|
||||
def index_video(fn, frame_type=None, cache_prefix=None):
|
||||
cache_path = cache_path_for_file_path(fn, cache_prefix)
|
||||
|
||||
if os.path.exists(cache_path):
|
||||
return
|
||||
|
||||
if frame_type is None:
|
||||
frame_type = fingerprint_video(fn[0])
|
||||
|
||||
if frame_type == FrameType.h265_stream:
|
||||
index_stream(fn, "hevc", cache_prefix=cache_prefix)
|
||||
else:
|
||||
raise NotImplementedError("Only h265 supported")
|
||||
|
||||
|
||||
def get_video_index(fn, frame_type, cache_prefix=None):
|
||||
cache_path = cache_path_for_file_path(fn, cache_prefix)
|
||||
|
||||
if not os.path.exists(cache_path):
|
||||
index_video(fn, frame_type, cache_prefix)
|
||||
|
||||
if not os.path.exists(cache_path):
|
||||
return None
|
||||
with open(cache_path, "rb") as cache_file:
|
||||
return pickle.load(cache_file)
|
||||
|
||||
|
||||
def read_file_check_size(f, sz, cookie):
|
||||
buff = bytearray(sz)
|
||||
bytes_read = f.readinto(buff)
|
||||
assert bytes_read == sz, (bytes_read, sz)
|
||||
return buff
|
||||
|
||||
|
||||
def rgb24toyuv(rgb):
|
||||
yuv_from_rgb = np.array([[ 0.299 , 0.587 , 0.114 ],
|
||||
[-0.14714119, -0.28886916, 0.43601035 ],
|
||||
[ 0.61497538, -0.51496512, -0.10001026 ]])
|
||||
img = np.dot(rgb.reshape(-1, 3), yuv_from_rgb.T).reshape(rgb.shape)
|
||||
|
||||
|
||||
|
||||
ys = img[:, :, 0]
|
||||
us = (img[::2, ::2, 1] + img[1::2, ::2, 1] + img[::2, 1::2, 1] + img[1::2, 1::2, 1]) / 4 + 128
|
||||
vs = (img[::2, ::2, 2] + img[1::2, ::2, 2] + img[::2, 1::2, 2] + img[1::2, 1::2, 2]) / 4 + 128
|
||||
|
||||
return ys, us, vs
|
||||
|
||||
|
||||
def rgb24toyuv420(rgb):
|
||||
ys, us, vs = rgb24toyuv(rgb)
|
||||
|
||||
y_len = rgb.shape[0] * rgb.shape[1]
|
||||
uv_len = y_len // 4
|
||||
|
||||
yuv420 = np.empty(y_len + 2 * uv_len, dtype=rgb.dtype)
|
||||
yuv420[:y_len] = ys.reshape(-1)
|
||||
yuv420[y_len:y_len + uv_len] = us.reshape(-1)
|
||||
yuv420[y_len + uv_len:y_len + 2 * uv_len] = vs.reshape(-1)
|
||||
|
||||
return yuv420.clip(0, 255).astype('uint8')
|
||||
|
||||
|
||||
def rgb24tonv12(rgb):
|
||||
ys, us, vs = rgb24toyuv(rgb)
|
||||
|
||||
y_len = rgb.shape[0] * rgb.shape[1]
|
||||
uv_len = y_len // 4
|
||||
|
||||
nv12 = np.empty(y_len + 2 * uv_len, dtype=rgb.dtype)
|
||||
nv12[:y_len] = ys.reshape(-1)
|
||||
nv12[y_len::2] = us.reshape(-1)
|
||||
nv12[y_len+1::2] = vs.reshape(-1)
|
||||
|
||||
return nv12.clip(0, 255).astype('uint8')
|
||||
|
||||
|
||||
def decompress_video_data(rawdat, vid_fmt, w, h, pix_fmt):
|
||||
# using a tempfile is much faster than proc.communicate for some reason
|
||||
|
||||
with tempfile.TemporaryFile() as tmpf:
|
||||
tmpf.write(rawdat)
|
||||
tmpf.seek(0)
|
||||
|
||||
threads = os.getenv("FFMPEG_THREADS", "0")
|
||||
cuda = os.getenv("FFMPEG_CUDA", "0") == "1"
|
||||
args = ["ffmpeg",
|
||||
"-threads", threads,
|
||||
"-hwaccel", "none" if not cuda else "cuda",
|
||||
"-c:v", "hevc",
|
||||
"-vsync", "0",
|
||||
"-f", vid_fmt,
|
||||
"-flags2", "showall",
|
||||
"-i", "pipe:0",
|
||||
"-threads", threads,
|
||||
"-f", "rawvideo",
|
||||
"-pix_fmt", pix_fmt,
|
||||
"pipe:1"]
|
||||
with subprocess.Popen(args, stdin=tmpf, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL) as proc:
|
||||
# dat = proc.communicate()[0]
|
||||
dat = proc.stdout.read()
|
||||
if proc.wait() != 0:
|
||||
raise DataUnreadableError("ffmpeg failed")
|
||||
|
||||
if pix_fmt == "rgb24":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, h, w, 3)
|
||||
elif pix_fmt == "nv12":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, (h*w*3//2))
|
||||
elif pix_fmt == "yuv420p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, (h*w*3//2))
|
||||
elif pix_fmt == "yuv444p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape(-1, 3, h, w)
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
class BaseFrameReader:
|
||||
# properties: frame_type, frame_count, w, h
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, *args):
|
||||
self.close()
|
||||
|
||||
def close(self):
|
||||
pass
|
||||
|
||||
def get(self, num, count=1, pix_fmt="yuv420p"):
|
||||
raise NotImplementedError
|
||||
|
||||
|
||||
def FrameReader(fn, cache_prefix=None, readahead=False, readbehind=False, index_data=None):
|
||||
frame_type = fingerprint_video(fn)
|
||||
if frame_type == FrameType.raw:
|
||||
return RawFrameReader(fn)
|
||||
elif frame_type in (FrameType.h265_stream,):
|
||||
if not index_data:
|
||||
index_data = get_video_index(fn, frame_type, cache_prefix)
|
||||
return StreamFrameReader(fn, frame_type, index_data, readahead=readahead, readbehind=readbehind)
|
||||
else:
|
||||
raise NotImplementedError(frame_type)
|
||||
|
||||
|
||||
class RawData:
|
||||
def __init__(self, f):
|
||||
self.f = _io.FileIO(f, 'rb')
|
||||
self.lenn = struct.unpack("I", self.f.read(4))[0]
|
||||
self.count = os.path.getsize(f) / (self.lenn+4)
|
||||
|
||||
def read(self, i):
|
||||
self.f.seek((self.lenn+4)*i + 4)
|
||||
return self.f.read(self.lenn)
|
||||
|
||||
|
||||
class RawFrameReader(BaseFrameReader):
|
||||
def __init__(self, fn):
|
||||
# raw camera
|
||||
self.fn = fn
|
||||
self.frame_type = FrameType.raw
|
||||
self.rawfile = RawData(self.fn)
|
||||
self.frame_count = self.rawfile.count
|
||||
self.w, self.h = 640, 480
|
||||
|
||||
def load_and_debayer(self, img):
|
||||
img = np.frombuffer(img, dtype='uint8').reshape(960, 1280)
|
||||
cimg = np.dstack([img[0::2, 1::2], ((img[0::2, 0::2].astype("uint16") + img[1::2, 1::2].astype("uint16")) >> 1).astype("uint8"), img[1::2, 0::2]])
|
||||
return cimg
|
||||
|
||||
def get(self, num, count=1, pix_fmt="yuv420p"):
|
||||
assert self.frame_count is not None
|
||||
assert num+count <= self.frame_count
|
||||
|
||||
if pix_fmt not in ("nv12", "yuv420p", "rgb24"):
|
||||
raise ValueError(f"Unsupported pixel format {pix_fmt!r}")
|
||||
|
||||
app = []
|
||||
for i in range(num, num+count):
|
||||
dat = self.rawfile.read(i)
|
||||
rgb_dat = self.load_and_debayer(dat)
|
||||
if pix_fmt == "rgb24":
|
||||
app.append(rgb_dat)
|
||||
elif pix_fmt == "nv12":
|
||||
app.append(rgb24tonv12(rgb_dat))
|
||||
elif pix_fmt == "yuv420p":
|
||||
app.append(rgb24toyuv420(rgb_dat))
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
return app
|
||||
|
||||
|
||||
class VideoStreamDecompressor:
|
||||
def __init__(self, fn, vid_fmt, w, h, pix_fmt):
|
||||
self.fn = fn
|
||||
self.vid_fmt = vid_fmt
|
||||
self.w = w
|
||||
self.h = h
|
||||
self.pix_fmt = pix_fmt
|
||||
|
||||
if pix_fmt in ("nv12", "yuv420p"):
|
||||
self.out_size = w*h*3//2 # yuv420p
|
||||
elif pix_fmt in ("rgb24", "yuv444p"):
|
||||
self.out_size = w*h*3
|
||||
else:
|
||||
raise NotImplementedError
|
||||
|
||||
self.proc = None
|
||||
self.t = threading.Thread(target=self.write_thread)
|
||||
self.t.daemon = True
|
||||
|
||||
def write_thread(self):
|
||||
try:
|
||||
with FileReader(self.fn) as f:
|
||||
while True:
|
||||
r = f.read(1024*1024)
|
||||
if len(r) == 0:
|
||||
break
|
||||
self.proc.stdin.write(r)
|
||||
except BrokenPipeError:
|
||||
pass
|
||||
finally:
|
||||
self.proc.stdin.close()
|
||||
|
||||
def read(self):
|
||||
threads = os.getenv("FFMPEG_THREADS", "0")
|
||||
cuda = os.getenv("FFMPEG_CUDA", "0") == "1"
|
||||
cmd = [
|
||||
"ffmpeg",
|
||||
"-threads", threads,
|
||||
"-hwaccel", "none" if not cuda else "cuda",
|
||||
"-c:v", "hevc",
|
||||
# "-avioflags", "direct",
|
||||
"-analyzeduration", "0",
|
||||
"-probesize", "32",
|
||||
"-flush_packets", "0",
|
||||
# "-fflags", "nobuffer",
|
||||
"-vsync", "0",
|
||||
"-f", self.vid_fmt,
|
||||
"-i", "pipe:0",
|
||||
"-threads", threads,
|
||||
"-f", "rawvideo",
|
||||
"-pix_fmt", self.pix_fmt,
|
||||
"pipe:1"
|
||||
]
|
||||
self.proc = subprocess.Popen(cmd, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL)
|
||||
try:
|
||||
self.t.start()
|
||||
|
||||
while True:
|
||||
dat = self.proc.stdout.read(self.out_size)
|
||||
if len(dat) == 0:
|
||||
break
|
||||
assert len(dat) == self.out_size
|
||||
if self.pix_fmt == "rgb24":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape((self.h, self.w, 3))
|
||||
elif self.pix_fmt == "yuv420p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8)
|
||||
elif self.pix_fmt == "nv12":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8)
|
||||
elif self.pix_fmt == "yuv444p":
|
||||
ret = np.frombuffer(dat, dtype=np.uint8).reshape((3, self.h, self.w))
|
||||
else:
|
||||
raise RuntimeError(f"unknown pix_fmt: {self.pix_fmt}")
|
||||
yield ret
|
||||
|
||||
result_code = self.proc.wait()
|
||||
assert result_code == 0, result_code
|
||||
finally:
|
||||
self.proc.kill()
|
||||
self.t.join()
|
||||
|
||||
class StreamGOPReader(GOPReader):
|
||||
def __init__(self, fn, frame_type, index_data):
|
||||
assert frame_type == FrameType.h265_stream
|
||||
|
||||
self.fn = fn
|
||||
|
||||
self.frame_type = frame_type
|
||||
self.frame_count = None
|
||||
self.w, self.h = None, None
|
||||
|
||||
self.prefix = None
|
||||
self.index = None
|
||||
|
||||
self.index = index_data['index']
|
||||
self.prefix = index_data['global_prefix']
|
||||
probe = index_data['probe']
|
||||
|
||||
self.prefix_frame_data = None
|
||||
self.num_prefix_frames = 0
|
||||
self.vid_fmt = "hevc"
|
||||
|
||||
i = 0
|
||||
while i < self.index.shape[0] and self.index[i, 0] != HEVC_SLICE_I:
|
||||
i += 1
|
||||
self.first_iframe = i
|
||||
|
||||
assert self.first_iframe == 0
|
||||
|
||||
self.frame_count = len(self.index) - 1
|
||||
|
||||
self.w = probe['streams'][0]['width']
|
||||
self.h = probe['streams'][0]['height']
|
||||
|
||||
def _lookup_gop(self, num):
|
||||
frame_b = num
|
||||
while frame_b > 0 and self.index[frame_b, 0] != HEVC_SLICE_I:
|
||||
frame_b -= 1
|
||||
|
||||
frame_e = num + 1
|
||||
while frame_e < (len(self.index) - 1) and self.index[frame_e, 0] != HEVC_SLICE_I:
|
||||
frame_e += 1
|
||||
|
||||
offset_b = self.index[frame_b, 1]
|
||||
offset_e = self.index[frame_e, 1]
|
||||
|
||||
return (frame_b, frame_e, offset_b, offset_e)
|
||||
|
||||
def get_gop(self, num):
|
||||
frame_b, frame_e, offset_b, offset_e = self._lookup_gop(num)
|
||||
assert frame_b <= num < frame_e
|
||||
|
||||
num_frames = frame_e - frame_b
|
||||
|
||||
with FileReader(self.fn) as f:
|
||||
f.seek(offset_b)
|
||||
rawdat = f.read(offset_e - offset_b)
|
||||
|
||||
if num < self.first_iframe:
|
||||
assert self.prefix_frame_data
|
||||
rawdat = self.prefix_frame_data + rawdat
|
||||
|
||||
rawdat = self.prefix + rawdat
|
||||
|
||||
skip_frames = 0
|
||||
if num < self.first_iframe:
|
||||
skip_frames = self.num_prefix_frames
|
||||
|
||||
return frame_b, num_frames, skip_frames, rawdat
|
||||
|
||||
|
||||
class GOPFrameReader(BaseFrameReader):
|
||||
#FrameReader with caching and readahead for formats that are group-of-picture based
|
||||
|
||||
def __init__(self, readahead=False, readbehind=False):
|
||||
self.open_ = True
|
||||
|
||||
self.readahead = readahead
|
||||
self.readbehind = readbehind
|
||||
self.frame_cache = LRU(64)
|
||||
|
||||
if self.readahead:
|
||||
self.cache_lock = threading.RLock()
|
||||
self.readahead_last = None
|
||||
self.readahead_len = 30
|
||||
self.readahead_c = threading.Condition()
|
||||
self.readahead_thread = threading.Thread(target=self._readahead_thread)
|
||||
self.readahead_thread.daemon = True
|
||||
self.readahead_thread.start()
|
||||
else:
|
||||
self.cache_lock = DoNothingContextManager()
|
||||
|
||||
def close(self):
|
||||
if not self.open_:
|
||||
return
|
||||
self.open_ = False
|
||||
|
||||
if self.readahead:
|
||||
self.readahead_c.acquire()
|
||||
self.readahead_c.notify()
|
||||
self.readahead_c.release()
|
||||
self.readahead_thread.join()
|
||||
|
||||
def _readahead_thread(self):
|
||||
while True:
|
||||
self.readahead_c.acquire()
|
||||
try:
|
||||
if not self.open_:
|
||||
break
|
||||
self.readahead_c.wait()
|
||||
finally:
|
||||
self.readahead_c.release()
|
||||
if not self.open_:
|
||||
break
|
||||
assert self.readahead_last
|
||||
num, pix_fmt = self.readahead_last
|
||||
|
||||
if self.readbehind:
|
||||
for k in range(num - 1, max(0, num - self.readahead_len), -1):
|
||||
self._get_one(k, pix_fmt)
|
||||
else:
|
||||
for k in range(num, min(self.frame_count, num + self.readahead_len)):
|
||||
self._get_one(k, pix_fmt)
|
||||
|
||||
def _get_one(self, num, pix_fmt):
|
||||
assert num < self.frame_count
|
||||
|
||||
if (num, pix_fmt) in self.frame_cache:
|
||||
return self.frame_cache[(num, pix_fmt)]
|
||||
|
||||
with self.cache_lock:
|
||||
if (num, pix_fmt) in self.frame_cache:
|
||||
return self.frame_cache[(num, pix_fmt)]
|
||||
|
||||
frame_b, num_frames, skip_frames, rawdat = self.get_gop(num)
|
||||
|
||||
ret = decompress_video_data(rawdat, self.vid_fmt, self.w, self.h, pix_fmt)
|
||||
ret = ret[skip_frames:]
|
||||
assert ret.shape[0] == num_frames
|
||||
|
||||
for i in range(ret.shape[0]):
|
||||
self.frame_cache[(frame_b+i, pix_fmt)] = ret[i]
|
||||
|
||||
return self.frame_cache[(num, pix_fmt)]
|
||||
|
||||
def get(self, num, count=1, pix_fmt="yuv420p"):
|
||||
assert self.frame_count is not None
|
||||
|
||||
if num + count > self.frame_count:
|
||||
raise ValueError(f"{num + count} > {self.frame_count}")
|
||||
|
||||
if pix_fmt not in ("nv12", "yuv420p", "rgb24", "yuv444p"):
|
||||
raise ValueError(f"Unsupported pixel format {pix_fmt!r}")
|
||||
|
||||
ret = [self._get_one(num + i, pix_fmt) for i in range(count)]
|
||||
|
||||
if self.readahead:
|
||||
self.readahead_last = (num+count, pix_fmt)
|
||||
self.readahead_c.acquire()
|
||||
self.readahead_c.notify()
|
||||
self.readahead_c.release()
|
||||
|
||||
return ret
|
||||
|
||||
|
||||
class StreamFrameReader(StreamGOPReader, GOPFrameReader):
|
||||
def __init__(self, fn, frame_type, index_data, readahead=False, readbehind=False):
|
||||
StreamGOPReader.__init__(self, fn, frame_type, index_data)
|
||||
GOPFrameReader.__init__(self, readahead, readbehind)
|
||||
|
||||
|
||||
def GOPFrameIterator(gop_reader, pix_fmt):
|
||||
dec = VideoStreamDecompressor(gop_reader.fn, gop_reader.vid_fmt, gop_reader.w, gop_reader.h, pix_fmt)
|
||||
yield from dec.read()
|
||||
|
||||
|
||||
def FrameIterator(fn, pix_fmt, **kwargs):
|
||||
fr = FrameReader(fn, **kwargs)
|
||||
if isinstance(fr, GOPReader):
|
||||
yield from GOPFrameIterator(fr, pix_fmt)
|
||||
else:
|
||||
for i in range(fr.frame_count):
|
||||
yield fr.get(i, pix_fmt=pix_fmt)[0]
|
||||
32
tools/lib/helpers.py
Normal file
32
tools/lib/helpers.py
Normal file
@@ -0,0 +1,32 @@
|
||||
import bz2
|
||||
import datetime
|
||||
|
||||
TIME_FMT = "%Y-%m-%d--%H-%M-%S"
|
||||
|
||||
# regex patterns
|
||||
class RE:
|
||||
DONGLE_ID = r'(?P<dongle_id>[a-z0-9]{16})'
|
||||
TIMESTAMP = r'(?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}--[0-9]{2}-[0-9]{2}-[0-9]{2})'
|
||||
ROUTE_NAME = r'{}[|_/]{}'.format(DONGLE_ID, TIMESTAMP)
|
||||
SEGMENT_NAME = r'{}(?:--|/)(?P<segment_num>[0-9]+)'.format(ROUTE_NAME)
|
||||
BOOTLOG_NAME = ROUTE_NAME
|
||||
|
||||
EXPLORER_FILE = r'^(?P<segment_name>{})--(?P<file_name>[a-z]+\.[a-z0-9]+)$'.format(SEGMENT_NAME)
|
||||
OP_SEGMENT_DIR = r'^(?P<segment_name>{})$'.format(SEGMENT_NAME)
|
||||
|
||||
|
||||
def timestamp_to_datetime(t: str) -> datetime.datetime:
|
||||
"""
|
||||
Convert an openpilot route timestamp to a python datetime
|
||||
"""
|
||||
return datetime.datetime.strptime(t, TIME_FMT)
|
||||
|
||||
|
||||
def save_log(dest, log_msgs, compress=True):
|
||||
dat = b"".join(msg.as_builder().to_bytes() for msg in log_msgs)
|
||||
|
||||
if compress:
|
||||
dat = bz2.compress(dat)
|
||||
|
||||
with open(dest, "wb") as f:
|
||||
f.write(dat)
|
||||
81
tools/lib/kbhit.py
Normal file
81
tools/lib/kbhit.py
Normal file
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python
|
||||
import sys
|
||||
import termios
|
||||
import atexit
|
||||
from select import select
|
||||
|
||||
STDIN_FD = sys.stdin.fileno()
|
||||
|
||||
class KBHit:
|
||||
def __init__(self) -> None:
|
||||
''' Creates a KBHit object that you can call to do various keyboard things.
|
||||
'''
|
||||
|
||||
self.set_kbhit_terminal()
|
||||
|
||||
def set_kbhit_terminal(self) -> None:
|
||||
''' Save old terminal settings for closure, remove ICANON & ECHO flags.
|
||||
'''
|
||||
|
||||
# Save the terminal settings
|
||||
self.old_term = termios.tcgetattr(STDIN_FD)
|
||||
self.new_term = self.old_term.copy()
|
||||
|
||||
# New terminal setting unbuffered
|
||||
self.new_term[3] &= ~(termios.ICANON | termios.ECHO)
|
||||
termios.tcsetattr(STDIN_FD, termios.TCSAFLUSH, self.new_term)
|
||||
|
||||
# Support normal-terminal reset at exit
|
||||
atexit.register(self.set_normal_term)
|
||||
|
||||
def set_normal_term(self) -> None:
|
||||
''' Resets to normal terminal. On Windows this is a no-op.
|
||||
'''
|
||||
|
||||
termios.tcsetattr(STDIN_FD, termios.TCSAFLUSH, self.old_term)
|
||||
|
||||
@staticmethod
|
||||
def getch() -> str:
|
||||
''' Returns a keyboard character after kbhit() has been called.
|
||||
Should not be called in the same program as getarrow().
|
||||
'''
|
||||
return sys.stdin.read(1)
|
||||
|
||||
@staticmethod
|
||||
def getarrow() -> int:
|
||||
''' Returns an arrow-key code after kbhit() has been called. Codes are
|
||||
0 : up
|
||||
1 : right
|
||||
2 : down
|
||||
3 : left
|
||||
Should not be called in the same program as getch().
|
||||
'''
|
||||
|
||||
c = sys.stdin.read(3)[2]
|
||||
vals = [65, 67, 66, 68]
|
||||
|
||||
return vals.index(ord(c))
|
||||
|
||||
@staticmethod
|
||||
def kbhit():
|
||||
''' Returns True if keyboard character was hit, False otherwise.
|
||||
'''
|
||||
return select([sys.stdin], [], [], 0)[0] != []
|
||||
|
||||
|
||||
# Test
|
||||
if __name__ == "__main__":
|
||||
|
||||
kb = KBHit()
|
||||
|
||||
print('Hit any key, or ESC to exit')
|
||||
|
||||
while True:
|
||||
|
||||
if kb.kbhit():
|
||||
c = kb.getch()
|
||||
if c == '\x1b': # ESC
|
||||
break
|
||||
print(c)
|
||||
|
||||
kb.set_normal_term()
|
||||
138
tools/lib/logreader.py
Executable file
138
tools/lib/logreader.py
Executable file
@@ -0,0 +1,138 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import sys
|
||||
import bz2
|
||||
import urllib.parse
|
||||
import capnp
|
||||
import warnings
|
||||
|
||||
|
||||
from cereal import log as capnp_log
|
||||
from openpilot.tools.lib.filereader import FileReader
|
||||
from openpilot.tools.lib.route import Route, SegmentName
|
||||
|
||||
# this is an iterator itself, and uses private variables from LogReader
|
||||
class MultiLogIterator:
|
||||
def __init__(self, log_paths, sort_by_time=False):
|
||||
self._log_paths = log_paths
|
||||
self.sort_by_time = sort_by_time
|
||||
|
||||
self._first_log_idx = next(i for i in range(len(log_paths)) if log_paths[i] is not None)
|
||||
self._current_log = self._first_log_idx
|
||||
self._idx = 0
|
||||
self._log_readers = [None]*len(log_paths)
|
||||
self.start_time = self._log_reader(self._first_log_idx)._ts[0]
|
||||
|
||||
def _log_reader(self, i):
|
||||
if self._log_readers[i] is None and self._log_paths[i] is not None:
|
||||
log_path = self._log_paths[i]
|
||||
self._log_readers[i] = LogReader(log_path, sort_by_time=self.sort_by_time)
|
||||
|
||||
return self._log_readers[i]
|
||||
|
||||
def __iter__(self):
|
||||
return self
|
||||
|
||||
def _inc(self):
|
||||
lr = self._log_reader(self._current_log)
|
||||
if self._idx < len(lr._ents)-1:
|
||||
self._idx += 1
|
||||
else:
|
||||
self._idx = 0
|
||||
self._current_log = next(i for i in range(self._current_log + 1, len(self._log_readers) + 1)
|
||||
if i == len(self._log_readers) or self._log_paths[i] is not None)
|
||||
if self._current_log == len(self._log_readers):
|
||||
raise StopIteration
|
||||
|
||||
def __next__(self):
|
||||
while 1:
|
||||
lr = self._log_reader(self._current_log)
|
||||
ret = lr._ents[self._idx]
|
||||
self._inc()
|
||||
return ret
|
||||
|
||||
def tell(self):
|
||||
# returns seconds from start of log
|
||||
return (self._log_reader(self._current_log)._ts[self._idx] - self.start_time) * 1e-9
|
||||
|
||||
def seek(self, ts):
|
||||
# seek to nearest minute
|
||||
minute = int(ts/60)
|
||||
if minute >= len(self._log_paths) or self._log_paths[minute] is None:
|
||||
return False
|
||||
|
||||
self._current_log = minute
|
||||
|
||||
# HACK: O(n) seek afterward
|
||||
self._idx = 0
|
||||
while self.tell() < ts:
|
||||
self._inc()
|
||||
return True
|
||||
|
||||
def reset(self):
|
||||
self.__init__(self._log_paths, sort_by_time=self.sort_by_time)
|
||||
|
||||
|
||||
class LogReader:
|
||||
def __init__(self, fn, canonicalize=True, only_union_types=False, sort_by_time=False, dat=None):
|
||||
self.data_version = None
|
||||
self._only_union_types = only_union_types
|
||||
|
||||
ext = None
|
||||
if not dat:
|
||||
_, ext = os.path.splitext(urllib.parse.urlparse(fn).path)
|
||||
if ext not in ('', '.bz2'):
|
||||
# old rlogs weren't bz2 compressed
|
||||
raise Exception(f"unknown extension {ext}")
|
||||
|
||||
with FileReader(fn) as f:
|
||||
dat = f.read()
|
||||
|
||||
if ext == ".bz2" or dat.startswith(b'BZh9'):
|
||||
dat = bz2.decompress(dat)
|
||||
|
||||
ents = capnp_log.Event.read_multiple_bytes(dat)
|
||||
|
||||
_ents = []
|
||||
try:
|
||||
for e in ents:
|
||||
_ents.append(e)
|
||||
except capnp.KjException:
|
||||
warnings.warn("Corrupted events detected", RuntimeWarning, stacklevel=1)
|
||||
|
||||
self._ents = list(sorted(_ents, key=lambda x: x.logMonoTime) if sort_by_time else _ents)
|
||||
self._ts = [x.logMonoTime for x in self._ents]
|
||||
|
||||
@classmethod
|
||||
def from_bytes(cls, dat):
|
||||
return cls("", dat=dat)
|
||||
|
||||
def __iter__(self):
|
||||
for ent in self._ents:
|
||||
if self._only_union_types:
|
||||
try:
|
||||
ent.which()
|
||||
yield ent
|
||||
except capnp.lib.capnp.KjException:
|
||||
pass
|
||||
else:
|
||||
yield ent
|
||||
|
||||
def logreader_from_route_or_segment(r, sort_by_time=False):
|
||||
sn = SegmentName(r, allow_route_name=True)
|
||||
route = Route(sn.route_name.canonical_name)
|
||||
if sn.segment_num < 0:
|
||||
return MultiLogIterator(route.log_paths(), sort_by_time=sort_by_time)
|
||||
else:
|
||||
return LogReader(route.log_paths()[sn.segment_num], sort_by_time=sort_by_time)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import codecs
|
||||
# capnproto <= 0.8.0 throws errors converting byte data to string
|
||||
# below line catches those errors and replaces the bytes with \x__
|
||||
codecs.register_error("strict", codecs.backslashreplace_errors)
|
||||
log_path = sys.argv[1]
|
||||
lr = LogReader(log_path, sort_by_time=True)
|
||||
for msg in lr:
|
||||
print(msg)
|
||||
231
tools/lib/route.py
Normal file
231
tools/lib/route.py
Normal file
@@ -0,0 +1,231 @@
|
||||
import os
|
||||
import re
|
||||
from urllib.parse import urlparse
|
||||
from collections import defaultdict
|
||||
from itertools import chain
|
||||
from typing import Optional
|
||||
|
||||
from openpilot.tools.lib.auth_config import get_token
|
||||
from openpilot.tools.lib.api import CommaApi
|
||||
from openpilot.tools.lib.helpers import RE
|
||||
|
||||
QLOG_FILENAMES = ['qlog', 'qlog.bz2']
|
||||
QCAMERA_FILENAMES = ['qcamera.ts']
|
||||
LOG_FILENAMES = ['rlog', 'rlog.bz2', 'raw_log.bz2']
|
||||
CAMERA_FILENAMES = ['fcamera.hevc', 'video.hevc']
|
||||
DCAMERA_FILENAMES = ['dcamera.hevc']
|
||||
ECAMERA_FILENAMES = ['ecamera.hevc']
|
||||
|
||||
class Route:
|
||||
def __init__(self, name, data_dir=None):
|
||||
self._name = RouteName(name)
|
||||
self.files = None
|
||||
if data_dir is not None:
|
||||
self._segments = self._get_segments_local(data_dir)
|
||||
else:
|
||||
self._segments = self._get_segments_remote()
|
||||
self.max_seg_number = self._segments[-1].name.segment_num
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self._name
|
||||
|
||||
@property
|
||||
def segments(self):
|
||||
return self._segments
|
||||
|
||||
def log_paths(self):
|
||||
log_path_by_seg_num = {s.name.segment_num: s.log_path for s in self._segments}
|
||||
return [log_path_by_seg_num.get(i, None) for i in range(self.max_seg_number+1)]
|
||||
|
||||
def qlog_paths(self):
|
||||
qlog_path_by_seg_num = {s.name.segment_num: s.qlog_path for s in self._segments}
|
||||
return [qlog_path_by_seg_num.get(i, None) for i in range(self.max_seg_number+1)]
|
||||
|
||||
def camera_paths(self):
|
||||
camera_path_by_seg_num = {s.name.segment_num: s.camera_path for s in self._segments}
|
||||
return [camera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number+1)]
|
||||
|
||||
def dcamera_paths(self):
|
||||
dcamera_path_by_seg_num = {s.name.segment_num: s.dcamera_path for s in self._segments}
|
||||
return [dcamera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number+1)]
|
||||
|
||||
def ecamera_paths(self):
|
||||
ecamera_path_by_seg_num = {s.name.segment_num: s.ecamera_path for s in self._segments}
|
||||
return [ecamera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number+1)]
|
||||
|
||||
def qcamera_paths(self):
|
||||
qcamera_path_by_seg_num = {s.name.segment_num: s.qcamera_path for s in self._segments}
|
||||
return [qcamera_path_by_seg_num.get(i, None) for i in range(self.max_seg_number+1)]
|
||||
|
||||
# TODO: refactor this, it's super repetitive
|
||||
def _get_segments_remote(self):
|
||||
api = CommaApi(get_token())
|
||||
route_files = api.get('v1/route/' + self.name.canonical_name + '/files')
|
||||
self.files = list(chain.from_iterable(route_files.values()))
|
||||
|
||||
segments = {}
|
||||
for url in self.files:
|
||||
_, dongle_id, time_str, segment_num, fn = urlparse(url).path.rsplit('/', maxsplit=4)
|
||||
segment_name = f'{dongle_id}|{time_str}--{segment_num}'
|
||||
if segments.get(segment_name):
|
||||
segments[segment_name] = Segment(
|
||||
segment_name,
|
||||
url if fn in LOG_FILENAMES else segments[segment_name].log_path,
|
||||
url if fn in QLOG_FILENAMES else segments[segment_name].qlog_path,
|
||||
url if fn in CAMERA_FILENAMES else segments[segment_name].camera_path,
|
||||
url if fn in DCAMERA_FILENAMES else segments[segment_name].dcamera_path,
|
||||
url if fn in ECAMERA_FILENAMES else segments[segment_name].ecamera_path,
|
||||
url if fn in QCAMERA_FILENAMES else segments[segment_name].qcamera_path,
|
||||
)
|
||||
else:
|
||||
segments[segment_name] = Segment(
|
||||
segment_name,
|
||||
url if fn in LOG_FILENAMES else None,
|
||||
url if fn in QLOG_FILENAMES else None,
|
||||
url if fn in CAMERA_FILENAMES else None,
|
||||
url if fn in DCAMERA_FILENAMES else None,
|
||||
url if fn in ECAMERA_FILENAMES else None,
|
||||
url if fn in QCAMERA_FILENAMES else None,
|
||||
)
|
||||
|
||||
return sorted(segments.values(), key=lambda seg: seg.name.segment_num)
|
||||
|
||||
def _get_segments_local(self, data_dir):
|
||||
files = os.listdir(data_dir)
|
||||
segment_files = defaultdict(list)
|
||||
|
||||
for f in files:
|
||||
fullpath = os.path.join(data_dir, f)
|
||||
explorer_match = re.match(RE.EXPLORER_FILE, f)
|
||||
op_match = re.match(RE.OP_SEGMENT_DIR, f)
|
||||
|
||||
if explorer_match:
|
||||
segment_name = explorer_match.group('segment_name')
|
||||
fn = explorer_match.group('file_name')
|
||||
if segment_name.replace('_', '|').startswith(self.name.canonical_name):
|
||||
segment_files[segment_name].append((fullpath, fn))
|
||||
elif op_match and os.path.isdir(fullpath):
|
||||
segment_name = op_match.group('segment_name')
|
||||
if segment_name.startswith(self.name.canonical_name):
|
||||
for seg_f in os.listdir(fullpath):
|
||||
segment_files[segment_name].append((os.path.join(fullpath, seg_f), seg_f))
|
||||
elif f == self.name.canonical_name:
|
||||
for seg_num in os.listdir(fullpath):
|
||||
if not seg_num.isdigit():
|
||||
continue
|
||||
|
||||
segment_name = f'{self.name.canonical_name}--{seg_num}'
|
||||
for seg_f in os.listdir(os.path.join(fullpath, seg_num)):
|
||||
segment_files[segment_name].append((os.path.join(fullpath, seg_num, seg_f), seg_f))
|
||||
|
||||
segments = []
|
||||
for segment, files in segment_files.items():
|
||||
|
||||
try:
|
||||
log_path = next(path for path, filename in files if filename in LOG_FILENAMES)
|
||||
except StopIteration:
|
||||
log_path = None
|
||||
|
||||
try:
|
||||
qlog_path = next(path for path, filename in files if filename in QLOG_FILENAMES)
|
||||
except StopIteration:
|
||||
qlog_path = None
|
||||
|
||||
try:
|
||||
camera_path = next(path for path, filename in files if filename in CAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
camera_path = None
|
||||
|
||||
try:
|
||||
dcamera_path = next(path for path, filename in files if filename in DCAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
dcamera_path = None
|
||||
|
||||
try:
|
||||
ecamera_path = next(path for path, filename in files if filename in ECAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
ecamera_path = None
|
||||
|
||||
try:
|
||||
qcamera_path = next(path for path, filename in files if filename in QCAMERA_FILENAMES)
|
||||
except StopIteration:
|
||||
qcamera_path = None
|
||||
|
||||
segments.append(Segment(segment, log_path, qlog_path, camera_path, dcamera_path, ecamera_path, qcamera_path))
|
||||
|
||||
if len(segments) == 0:
|
||||
raise ValueError(f'Could not find segments for route {self.name.canonical_name} in data directory {data_dir}')
|
||||
return sorted(segments, key=lambda seg: seg.name.segment_num)
|
||||
|
||||
class Segment:
|
||||
def __init__(self, name, log_path, qlog_path, camera_path, dcamera_path, ecamera_path, qcamera_path):
|
||||
self._name = SegmentName(name)
|
||||
self.log_path = log_path
|
||||
self.qlog_path = qlog_path
|
||||
self.camera_path = camera_path
|
||||
self.dcamera_path = dcamera_path
|
||||
self.ecamera_path = ecamera_path
|
||||
self.qcamera_path = qcamera_path
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
return self._name
|
||||
|
||||
class RouteName:
|
||||
def __init__(self, name_str: str):
|
||||
self._name_str = name_str
|
||||
delim = next(c for c in self._name_str if c in ("|", "/"))
|
||||
self._dongle_id, self._time_str = self._name_str.split(delim)
|
||||
|
||||
assert len(self._dongle_id) == 16, self._name_str
|
||||
assert len(self._time_str) == 20, self._name_str
|
||||
self._canonical_name = f"{self._dongle_id}|{self._time_str}"
|
||||
|
||||
@property
|
||||
def canonical_name(self) -> str: return self._canonical_name
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str: return self._dongle_id
|
||||
|
||||
@property
|
||||
def time_str(self) -> str: return self._time_str
|
||||
|
||||
def __str__(self) -> str: return self._canonical_name
|
||||
|
||||
class SegmentName:
|
||||
# TODO: add constructor that takes dongle_id, time_str, segment_num and then create instances
|
||||
# of this class instead of manually constructing a segment name (use canonical_name prop instead)
|
||||
def __init__(self, name_str: str, allow_route_name=False):
|
||||
data_dir_path_separator_index = name_str.rsplit("|", 1)[0].rfind("/")
|
||||
use_data_dir = (data_dir_path_separator_index != -1) and ("|" in name_str)
|
||||
self._name_str = name_str[data_dir_path_separator_index + 1:] if use_data_dir else name_str
|
||||
self._data_dir = name_str[:data_dir_path_separator_index] if use_data_dir else None
|
||||
|
||||
seg_num_delim = "--" if self._name_str.count("--") == 2 else "/"
|
||||
name_parts = self._name_str.rsplit(seg_num_delim, 1)
|
||||
if allow_route_name and len(name_parts) == 1:
|
||||
name_parts.append("-1") # no segment number
|
||||
self._route_name = RouteName(name_parts[0])
|
||||
self._num = int(name_parts[1])
|
||||
self._canonical_name = f"{self._route_name._dongle_id}|{self._route_name._time_str}--{self._num}"
|
||||
|
||||
@property
|
||||
def canonical_name(self) -> str: return self._canonical_name
|
||||
|
||||
@property
|
||||
def dongle_id(self) -> str: return self._route_name.dongle_id
|
||||
|
||||
@property
|
||||
def time_str(self) -> str: return self._route_name.time_str
|
||||
|
||||
@property
|
||||
def segment_num(self) -> int: return self._num
|
||||
|
||||
@property
|
||||
def route_name(self) -> RouteName: return self._route_name
|
||||
|
||||
@property
|
||||
def data_dir(self) -> Optional[str]: return self._data_dir
|
||||
|
||||
def __str__(self) -> str: return self._canonical_name
|
||||
0
tools/lib/tests/__init__.py
Normal file
0
tools/lib/tests/__init__.py
Normal file
69
tools/lib/tests/test_caching.py
Normal file
69
tools/lib/tests/test_caching.py
Normal file
@@ -0,0 +1,69 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import shutil
|
||||
import unittest
|
||||
|
||||
os.environ["COMMA_CACHE"] = "/tmp/__test_cache__"
|
||||
from openpilot.tools.lib.url_file import URLFile, CACHE_DIR
|
||||
|
||||
|
||||
class TestFileDownload(unittest.TestCase):
|
||||
|
||||
def compare_loads(self, url, start=0, length=None):
|
||||
"""Compares range between cached and non cached version"""
|
||||
shutil.rmtree(CACHE_DIR)
|
||||
|
||||
file_cached = URLFile(url, cache=True)
|
||||
file_downloaded = URLFile(url, cache=False)
|
||||
|
||||
file_cached.seek(start)
|
||||
file_downloaded.seek(start)
|
||||
|
||||
self.assertEqual(file_cached.get_length(), file_downloaded.get_length())
|
||||
self.assertLessEqual(length + start if length is not None else 0, file_downloaded.get_length())
|
||||
|
||||
response_cached = file_cached.read(ll=length)
|
||||
response_downloaded = file_downloaded.read(ll=length)
|
||||
|
||||
self.assertEqual(response_cached, response_downloaded)
|
||||
|
||||
# Now test with cache in place
|
||||
file_cached = URLFile(url, cache=True)
|
||||
file_cached.seek(start)
|
||||
response_cached = file_cached.read(ll=length)
|
||||
|
||||
self.assertEqual(file_cached.get_length(), file_downloaded.get_length())
|
||||
self.assertEqual(response_cached, response_downloaded)
|
||||
|
||||
def test_small_file(self):
|
||||
# Make sure we don't force cache
|
||||
os.environ["FILEREADER_CACHE"] = "0"
|
||||
small_file_url = "https://raw.githubusercontent.com/commaai/openpilot/master/docs/SAFETY.md"
|
||||
# If you want large file to be larger than a chunk
|
||||
# large_file_url = "https://commadataci.blob.core.windows.net/openpilotci/0375fdf7b1ce594d/2019-06-13--08-32-25/3/fcamera.hevc"
|
||||
|
||||
# Load full small file
|
||||
self.compare_loads(small_file_url)
|
||||
|
||||
file_small = URLFile(small_file_url)
|
||||
length = file_small.get_length()
|
||||
|
||||
self.compare_loads(small_file_url, length - 100, 100)
|
||||
self.compare_loads(small_file_url, 50, 100)
|
||||
|
||||
# Load small file 100 bytes at a time
|
||||
for i in range(length // 100):
|
||||
self.compare_loads(small_file_url, 100 * i, 100)
|
||||
|
||||
def test_large_file(self):
|
||||
large_file_url = "https://commadataci.blob.core.windows.net/openpilotci/0375fdf7b1ce594d/2019-06-13--08-32-25/3/qlog.bz2"
|
||||
# Load the end 100 bytes of both files
|
||||
file_large = URLFile(large_file_url)
|
||||
length = file_large.get_length()
|
||||
|
||||
self.compare_loads(large_file_url, length - 100, 100)
|
||||
self.compare_loads(large_file_url)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
67
tools/lib/tests/test_readers.py
Executable file
67
tools/lib/tests/test_readers.py
Executable file
@@ -0,0 +1,67 @@
|
||||
#!/usr/bin/env python
|
||||
import unittest
|
||||
import requests
|
||||
import tempfile
|
||||
|
||||
from collections import defaultdict
|
||||
import numpy as np
|
||||
from openpilot.tools.lib.framereader import FrameReader
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
|
||||
|
||||
class TestReaders(unittest.TestCase):
|
||||
@unittest.skip("skip for bandwidth reasons")
|
||||
def test_logreader(self):
|
||||
def _check_data(lr):
|
||||
hist = defaultdict(int)
|
||||
for l in lr:
|
||||
hist[l.which()] += 1
|
||||
|
||||
self.assertEqual(hist['carControl'], 6000)
|
||||
self.assertEqual(hist['logMessage'], 6857)
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".bz2") as fp:
|
||||
r = requests.get("https://github.com/commaai/comma2k19/blob/master/Example_1/b0c9d2329ad1606b%7C2018-08-02--08-34-47/40/raw_log.bz2?raw=true", timeout=10)
|
||||
fp.write(r.content)
|
||||
fp.flush()
|
||||
|
||||
lr_file = LogReader(fp.name)
|
||||
_check_data(lr_file)
|
||||
|
||||
lr_url = LogReader("https://github.com/commaai/comma2k19/blob/master/Example_1/b0c9d2329ad1606b%7C2018-08-02--08-34-47/40/raw_log.bz2?raw=true")
|
||||
_check_data(lr_url)
|
||||
|
||||
@unittest.skip("skip for bandwidth reasons")
|
||||
def test_framereader(self):
|
||||
def _check_data(f):
|
||||
self.assertEqual(f.frame_count, 1200)
|
||||
self.assertEqual(f.w, 1164)
|
||||
self.assertEqual(f.h, 874)
|
||||
|
||||
frame_first_30 = f.get(0, 30)
|
||||
self.assertEqual(len(frame_first_30), 30)
|
||||
|
||||
print(frame_first_30[15])
|
||||
|
||||
print("frame_0")
|
||||
frame_0 = f.get(0, 1)
|
||||
frame_15 = f.get(15, 1)
|
||||
|
||||
print(frame_15[0])
|
||||
|
||||
assert np.all(frame_first_30[0] == frame_0[0])
|
||||
assert np.all(frame_first_30[15] == frame_15[0])
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix=".hevc") as fp:
|
||||
r = requests.get("https://github.com/commaai/comma2k19/blob/master/Example_1/b0c9d2329ad1606b%7C2018-08-02--08-34-47/40/video.hevc?raw=true", timeout=10)
|
||||
fp.write(r.content)
|
||||
fp.flush()
|
||||
|
||||
fr_file = FrameReader(fp.name)
|
||||
_check_data(fr_file)
|
||||
|
||||
fr_url = FrameReader("https://github.com/commaai/comma2k19/blob/master/Example_1/b0c9d2329ad1606b%7C2018-08-02--08-34-47/40/video.hevc?raw=true")
|
||||
_check_data(fr_url)
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
32
tools/lib/tests/test_route_library.py
Executable file
32
tools/lib/tests/test_route_library.py
Executable file
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env python
|
||||
import unittest
|
||||
from collections import namedtuple
|
||||
|
||||
from openpilot.tools.lib.route import SegmentName
|
||||
|
||||
class TestRouteLibrary(unittest.TestCase):
|
||||
def test_segment_name_formats(self):
|
||||
Case = namedtuple('Case', ['input', 'expected_route', 'expected_segment_num', 'expected_data_dir'])
|
||||
|
||||
cases = [ Case("a2a0ccea32023010|2023-07-27--13-01-19", "a2a0ccea32023010|2023-07-27--13-01-19", -1, None),
|
||||
Case("a2a0ccea32023010/2023-07-27--13-01-19--1", "a2a0ccea32023010|2023-07-27--13-01-19", 1, None),
|
||||
Case("a2a0ccea32023010|2023-07-27--13-01-19/2", "a2a0ccea32023010|2023-07-27--13-01-19", 2, None),
|
||||
Case("a2a0ccea32023010/2023-07-27--13-01-19/3", "a2a0ccea32023010|2023-07-27--13-01-19", 3, None),
|
||||
Case("/data/media/0/realdata/a2a0ccea32023010|2023-07-27--13-01-19", "a2a0ccea32023010|2023-07-27--13-01-19", -1, "/data/media/0/realdata"),
|
||||
Case("/data/media/0/realdata/a2a0ccea32023010|2023-07-27--13-01-19--1", "a2a0ccea32023010|2023-07-27--13-01-19", 1, "/data/media/0/realdata"),
|
||||
Case("/data/media/0/realdata/a2a0ccea32023010|2023-07-27--13-01-19/2", "a2a0ccea32023010|2023-07-27--13-01-19", 2, "/data/media/0/realdata") ]
|
||||
|
||||
def _validate(case):
|
||||
route_or_segment_name = case.input
|
||||
|
||||
s = SegmentName(route_or_segment_name, allow_route_name=True)
|
||||
|
||||
self.assertEqual(str(s.route_name), case.expected_route)
|
||||
self.assertEqual(s.segment_num, case.expected_segment_num)
|
||||
self.assertEqual(s.data_dir, case.expected_data_dir)
|
||||
|
||||
for case in cases:
|
||||
_validate(case)
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
199
tools/lib/url_file.py
Normal file
199
tools/lib/url_file.py
Normal file
@@ -0,0 +1,199 @@
|
||||
# pylint: skip-file
|
||||
|
||||
import os
|
||||
import time
|
||||
import tempfile
|
||||
import threading
|
||||
import urllib.parse
|
||||
import pycurl
|
||||
from hashlib import sha256
|
||||
from io import BytesIO
|
||||
from tenacity import retry, wait_random_exponential, stop_after_attempt
|
||||
from openpilot.common.file_helpers import mkdirs_exists_ok, atomic_write_in_dir
|
||||
# Cache chunk size
|
||||
K = 1000
|
||||
CHUNK_SIZE = 1000 * K
|
||||
|
||||
CACHE_DIR = os.environ.get("COMMA_CACHE", "/tmp/comma_download_cache/")
|
||||
|
||||
|
||||
def hash_256(link):
|
||||
hsh = str(sha256((link.split("?")[0]).encode('utf-8')).hexdigest())
|
||||
return hsh
|
||||
|
||||
|
||||
class URLFile:
|
||||
_tlocal = threading.local()
|
||||
|
||||
def __init__(self, url, debug=False, cache=None):
|
||||
self._url = url
|
||||
self._pos = 0
|
||||
self._length = None
|
||||
self._local_file = None
|
||||
self._debug = debug
|
||||
# True by default, false if FILEREADER_CACHE is defined, but can be overwritten by the cache input
|
||||
self._force_download = not int(os.environ.get("FILEREADER_CACHE", "0"))
|
||||
if cache is not None:
|
||||
self._force_download = not cache
|
||||
|
||||
try:
|
||||
self._curl = self._tlocal.curl
|
||||
except AttributeError:
|
||||
self._curl = self._tlocal.curl = pycurl.Curl()
|
||||
mkdirs_exists_ok(CACHE_DIR)
|
||||
|
||||
def __enter__(self):
|
||||
return self
|
||||
|
||||
def __exit__(self, exc_type, exc_value, traceback):
|
||||
if self._local_file is not None:
|
||||
os.remove(self._local_file.name)
|
||||
self._local_file.close()
|
||||
self._local_file = None
|
||||
|
||||
@retry(wait=wait_random_exponential(multiplier=1, max=5), stop=stop_after_attempt(3), reraise=True)
|
||||
def get_length_online(self):
|
||||
c = self._curl
|
||||
c.reset()
|
||||
c.setopt(pycurl.NOSIGNAL, 1)
|
||||
c.setopt(pycurl.TIMEOUT_MS, 500000)
|
||||
c.setopt(pycurl.FOLLOWLOCATION, True)
|
||||
c.setopt(pycurl.URL, self._url)
|
||||
c.setopt(c.NOBODY, 1)
|
||||
c.perform()
|
||||
length = int(c.getinfo(c.CONTENT_LENGTH_DOWNLOAD))
|
||||
c.reset()
|
||||
return length
|
||||
|
||||
def get_length(self):
|
||||
if self._length is not None:
|
||||
return self._length
|
||||
file_length_path = os.path.join(CACHE_DIR, hash_256(self._url) + "_length")
|
||||
if os.path.exists(file_length_path) and not self._force_download:
|
||||
with open(file_length_path) as file_length:
|
||||
content = file_length.read()
|
||||
self._length = int(content)
|
||||
return self._length
|
||||
|
||||
self._length = self.get_length_online()
|
||||
if not self._force_download:
|
||||
with atomic_write_in_dir(file_length_path, mode="w") as file_length:
|
||||
file_length.write(str(self._length))
|
||||
return self._length
|
||||
|
||||
def read(self, ll=None):
|
||||
if self._force_download:
|
||||
return self.read_aux(ll=ll)
|
||||
|
||||
file_begin = self._pos
|
||||
file_end = self._pos + ll if ll is not None else self.get_length()
|
||||
assert file_end != -1, f"Remote file is empty or doesn't exist: {self._url}"
|
||||
# We have to align with chunks we store. Position is the begginiing of the latest chunk that starts before or at our file
|
||||
position = (file_begin // CHUNK_SIZE) * CHUNK_SIZE
|
||||
response = b""
|
||||
while True:
|
||||
self._pos = position
|
||||
chunk_number = self._pos / CHUNK_SIZE
|
||||
file_name = hash_256(self._url) + "_" + str(chunk_number)
|
||||
full_path = os.path.join(CACHE_DIR, str(file_name))
|
||||
data = None
|
||||
# If we don't have a file, download it
|
||||
if not os.path.exists(full_path):
|
||||
data = self.read_aux(ll=CHUNK_SIZE)
|
||||
with atomic_write_in_dir(full_path, mode="wb") as new_cached_file:
|
||||
new_cached_file.write(data)
|
||||
else:
|
||||
with open(full_path, "rb") as cached_file:
|
||||
data = cached_file.read()
|
||||
|
||||
response += data[max(0, file_begin - position): min(CHUNK_SIZE, file_end - position)]
|
||||
|
||||
position += CHUNK_SIZE
|
||||
if position >= file_end:
|
||||
self._pos = file_end
|
||||
return response
|
||||
|
||||
@retry(wait=wait_random_exponential(multiplier=1, max=5), stop=stop_after_attempt(3), reraise=True)
|
||||
def read_aux(self, ll=None):
|
||||
download_range = False
|
||||
headers = ["Connection: keep-alive"]
|
||||
if self._pos != 0 or ll is not None:
|
||||
if ll is None:
|
||||
end = self.get_length() - 1
|
||||
else:
|
||||
end = min(self._pos + ll, self.get_length()) - 1
|
||||
if self._pos >= end:
|
||||
return b""
|
||||
headers.append(f"Range: bytes={self._pos}-{end}")
|
||||
download_range = True
|
||||
|
||||
dats = BytesIO()
|
||||
c = self._curl
|
||||
c.setopt(pycurl.URL, self._url)
|
||||
c.setopt(pycurl.WRITEDATA, dats)
|
||||
c.setopt(pycurl.NOSIGNAL, 1)
|
||||
c.setopt(pycurl.TIMEOUT_MS, 500000)
|
||||
c.setopt(pycurl.HTTPHEADER, headers)
|
||||
c.setopt(pycurl.FOLLOWLOCATION, True)
|
||||
|
||||
if self._debug:
|
||||
print("downloading", self._url)
|
||||
|
||||
def header(x):
|
||||
if b'MISS' in x:
|
||||
print(x.strip())
|
||||
|
||||
c.setopt(pycurl.HEADERFUNCTION, header)
|
||||
|
||||
def test(debug_type, debug_msg):
|
||||
print(" debug(%d): %s" % (debug_type, debug_msg.strip()))
|
||||
|
||||
c.setopt(pycurl.VERBOSE, 1)
|
||||
c.setopt(pycurl.DEBUGFUNCTION, test)
|
||||
t1 = time.time()
|
||||
|
||||
c.perform()
|
||||
|
||||
if self._debug:
|
||||
t2 = time.time()
|
||||
if t2 - t1 > 0.1:
|
||||
print(f"get {self._url} {headers!r} {t2 - t1:.f} slow")
|
||||
|
||||
response_code = c.getinfo(pycurl.RESPONSE_CODE)
|
||||
if response_code == 416: # Requested Range Not Satisfiable
|
||||
raise Exception(f"Error, range out of bounds {response_code} {headers} ({self._url}): {repr(dats.getvalue())[:500]}")
|
||||
if download_range and response_code != 206: # Partial Content
|
||||
raise Exception(f"Error, requested range but got unexpected response {response_code} {headers} ({self._url}): {repr(dats.getvalue())[:500]}")
|
||||
if (not download_range) and response_code != 200: # OK
|
||||
raise Exception(f"Error {response_code} {headers} ({self._url}): {repr(dats.getvalue())[:500]}")
|
||||
|
||||
ret = dats.getvalue()
|
||||
self._pos += len(ret)
|
||||
return ret
|
||||
|
||||
def seek(self, pos):
|
||||
self._pos = pos
|
||||
|
||||
@property
|
||||
def name(self):
|
||||
"""Returns a local path to file with the URLFile's contents.
|
||||
|
||||
This can be used to interface with modules that require local files.
|
||||
"""
|
||||
if self._local_file is None:
|
||||
_, ext = os.path.splitext(urllib.parse.urlparse(self._url).path)
|
||||
local_fd, local_path = tempfile.mkstemp(suffix=ext)
|
||||
try:
|
||||
os.write(local_fd, self.read())
|
||||
local_file = open(local_path, "rb")
|
||||
except Exception:
|
||||
os.remove(local_path)
|
||||
raise
|
||||
finally:
|
||||
os.close(local_fd)
|
||||
|
||||
self._local_file = local_file
|
||||
self.read = self._local_file.read
|
||||
self.seek = self._local_file.seek
|
||||
|
||||
return self._local_file.name
|
||||
6
tools/lib/vidindex/Makefile
Normal file
6
tools/lib/vidindex/Makefile
Normal file
@@ -0,0 +1,6 @@
|
||||
CC := gcc
|
||||
|
||||
vidindex: bitstream.c bitstream.h vidindex.c
|
||||
$(eval $@_TMP := $(shell mktemp))
|
||||
$(CC) -std=c99 bitstream.c vidindex.c -o $($@_TMP)
|
||||
mv $($@_TMP) $@
|
||||
26
tools/lib/vidindex/bitstream.h
Normal file
26
tools/lib/vidindex/bitstream.h
Normal file
@@ -0,0 +1,26 @@
|
||||
#ifndef bitstream_H
|
||||
#define bitstream_H
|
||||
|
||||
|
||||
#include <stddef.h>
|
||||
#include <stdint.h>
|
||||
|
||||
struct bitstream {
|
||||
const uint8_t *buffer_ptr;
|
||||
const uint8_t *buffer_end;
|
||||
uint64_t value;
|
||||
uint32_t pos;
|
||||
uint32_t shift;
|
||||
size_t size;
|
||||
};
|
||||
|
||||
void bs_init(struct bitstream *bs, const uint8_t *buffer, size_t input_size);
|
||||
void bs_seek(struct bitstream *bs, size_t new_pos);
|
||||
uint32_t bs_get(struct bitstream *bs, int n);
|
||||
uint32_t bs_peek(struct bitstream *bs, int n);
|
||||
size_t bs_remain(struct bitstream *bs);
|
||||
int bs_eof(struct bitstream *bs);
|
||||
uint32_t bs_ue(struct bitstream *bs);
|
||||
int32_t bs_se(struct bitstream *bs);
|
||||
|
||||
#endif
|
||||
80
tools/mac_setup.sh
Executable file
80
tools/mac_setup.sh
Executable file
@@ -0,0 +1,80 @@
|
||||
#!/bin/bash
|
||||
|
||||
set -e
|
||||
|
||||
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null && pwd )"
|
||||
ROOT="$(cd $DIR/../ && pwd)"
|
||||
ARCH=$(uname -m)
|
||||
|
||||
if [[ $SHELL == "/bin/zsh" ]]; then
|
||||
RC_FILE="$HOME/.zshrc"
|
||||
elif [[ $SHELL == "/bin/bash" ]]; then
|
||||
RC_FILE="$HOME/.bash_profile"
|
||||
fi
|
||||
|
||||
# Install brew if required
|
||||
if [[ $(command -v brew) == "" ]]; then
|
||||
echo "Installing Hombrew"
|
||||
/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install.sh)"
|
||||
echo "[ ] installed brew t=$SECONDS"
|
||||
|
||||
# make brew available now
|
||||
if [[ $ARCH == "x86_64" ]]; then
|
||||
echo 'eval "$(/usr/local/homebrew/bin/brew shellenv)"' >> $RC_FILE
|
||||
eval "$(/usr/local/homebrew/bin/brew shellenv)"
|
||||
else
|
||||
echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> $RC_FILE
|
||||
eval "$(/opt/homebrew/bin/brew shellenv)"
|
||||
fi
|
||||
fi
|
||||
|
||||
brew bundle --file=- <<-EOS
|
||||
brew "catch2"
|
||||
brew "cmake"
|
||||
brew "cppcheck"
|
||||
brew "git-lfs"
|
||||
brew "zlib"
|
||||
brew "bzip2"
|
||||
brew "capnp"
|
||||
brew "coreutils"
|
||||
brew "eigen"
|
||||
brew "ffmpeg"
|
||||
brew "glfw"
|
||||
brew "libarchive"
|
||||
brew "libusb"
|
||||
brew "libtool"
|
||||
brew "llvm"
|
||||
brew "openssl@3.0"
|
||||
brew "pyenv"
|
||||
brew "pyenv-virtualenv"
|
||||
brew "qt@5"
|
||||
brew "zeromq"
|
||||
brew "gcc@12"
|
||||
cask "gcc-arm-embedded"
|
||||
brew "portaudio"
|
||||
EOS
|
||||
|
||||
echo "[ ] finished brew install t=$SECONDS"
|
||||
|
||||
BREW_PREFIX=$(brew --prefix)
|
||||
|
||||
# archive backend tools for pip dependencies
|
||||
export LDFLAGS="$LDFLAGS -L${BREW_PREFIX}/opt/zlib/lib"
|
||||
export LDFLAGS="$LDFLAGS -L${BREW_PREFIX}/opt/bzip2/lib"
|
||||
export CPPFLAGS="$CPPFLAGS -I${BREW_PREFIX}/opt/zlib/include"
|
||||
export CPPFLAGS="$CPPFLAGS -I${BREW_PREFIX}/opt/bzip2/include"
|
||||
|
||||
# pycurl curl/openssl backend dependencies
|
||||
export LDFLAGS="$LDFLAGS -L${BREW_PREFIX}/opt/openssl@3/lib"
|
||||
export CPPFLAGS="$CPPFLAGS -I${BREW_PREFIX}/opt/openssl@3/include"
|
||||
export PYCURL_CURL_CONFIG=/usr/bin/curl-config
|
||||
export PYCURL_SSL_LIBRARY=openssl
|
||||
|
||||
# install python dependencies
|
||||
$DIR/install_python_dependencies.sh
|
||||
echo "[ ] installed python dependencies t=$SECONDS"
|
||||
|
||||
echo
|
||||
echo "---- OPENPILOT SETUP DONE ----"
|
||||
echo "Open a new shell or configure your active shell env by running:"
|
||||
echo "source $RC_FILE"
|
||||
74
tools/plotjuggler/README.md
Normal file
74
tools/plotjuggler/README.md
Normal file
@@ -0,0 +1,74 @@
|
||||
# PlotJuggler
|
||||
|
||||
[PlotJuggler](https://github.com/facontidavide/PlotJuggler) is a tool to quickly visualize time series data, and we've written plugins to parse openpilot logs. Check out our plugins: https://github.com/commaai/PlotJuggler.
|
||||
|
||||
## Installation
|
||||
|
||||
Once you've [set up the openpilot environment](../README.md), this command will download PlotJuggler and install our plugins:
|
||||
|
||||
`cd tools/plotjuggler && ./juggle.py --install`
|
||||
|
||||
## Usage
|
||||
|
||||
```
|
||||
$ ./juggle.py -h
|
||||
usage: juggle.py [-h] [--demo] [--qlog] [--ci] [--can] [--stream] [--layout [LAYOUT]] [--install] [--dbc DBC]
|
||||
[route_or_segment_name] [segment_count]
|
||||
|
||||
A helper to run PlotJuggler on openpilot routes
|
||||
|
||||
positional arguments:
|
||||
route_or_segment_name
|
||||
The route or segment name to plot (cabana share URL accepted) (default: None)
|
||||
segment_count The number of segments to plot (default: None)
|
||||
|
||||
optional arguments:
|
||||
-h, --help show this help message and exit
|
||||
--demo Use the demo route instead of providing one (default: False)
|
||||
--qlog Use qlogs (default: False)
|
||||
--ci Download data from openpilot CI bucket (default: False)
|
||||
--can Parse CAN data (default: False)
|
||||
--stream Start PlotJuggler in streaming mode (default: False)
|
||||
--layout [LAYOUT] Run PlotJuggler with a pre-defined layout (default: None)
|
||||
--install Install or update PlotJuggler + plugins (default: False)
|
||||
--dbc DBC Set the DBC name to load for parsing CAN data. If not set, the DBC will be automatically
|
||||
inferred from the logs. (default: None)
|
||||
|
||||
```
|
||||
|
||||
Examples using route name:
|
||||
|
||||
`./juggle.py "a2a0ccea32023010|2023-07-27--13-01-19"`
|
||||
|
||||
Examples using segment name:
|
||||
|
||||
`./juggle.py "a2a0ccea32023010|2023-07-27--13-01-19--1"`
|
||||
|
||||
## Streaming
|
||||
|
||||
Explore live data from your car! Follow these steps to stream from your comma device to your laptop:
|
||||
- Enable wifi tethering on your comma device
|
||||
- [SSH into your device](https://github.com/commaai/openpilot/wiki/SSH) and run `cd /data/openpilot && ./cereal/messaging/bridge`
|
||||
- On your laptop, connect to the device's wifi hotspot
|
||||
- Start PlotJuggler with `ZMQ=1 ./juggle.py --stream`, find the `Cereal Subscriber` plugin in the dropdown under Streaming, and click `Start`.
|
||||
|
||||
If streaming to PlotJuggler from a replay on your PC, simply run: `./juggle.py --stream` and start the cereal subscriber.
|
||||
|
||||
## Demo
|
||||
|
||||
For a quick demo, go through the installation step and run this command:
|
||||
|
||||
`./juggle.py --demo --qlog --layout=layouts/demo.xml`
|
||||
|
||||
## Layouts
|
||||
|
||||
If you create a layout that's useful for others, consider upstreaming it.
|
||||
|
||||
### Tuning
|
||||
|
||||
Use this layout to improve your car's tuning and generate plots for tuning PRs. Also see the [tuning wiki](https://github.com/commaai/openpilot/wiki/Tuning) and tuning PR template.
|
||||
|
||||
`--layout layouts/tuning.xml`
|
||||
|
||||
|
||||
|
||||
180
tools/plotjuggler/juggle.py
Executable file
180
tools/plotjuggler/juggle.py
Executable file
@@ -0,0 +1,180 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import sys
|
||||
import multiprocessing
|
||||
import platform
|
||||
import shutil
|
||||
import subprocess
|
||||
import tarfile
|
||||
import tempfile
|
||||
import requests
|
||||
import argparse
|
||||
|
||||
from openpilot.common.basedir import BASEDIR
|
||||
from openpilot.selfdrive.test.openpilotci import get_url
|
||||
from openpilot.tools.lib.logreader import LogReader
|
||||
from openpilot.tools.lib.route import Route, SegmentName
|
||||
from openpilot.tools.lib.helpers import save_log
|
||||
from urllib.parse import urlparse, parse_qs
|
||||
|
||||
juggle_dir = os.path.dirname(os.path.realpath(__file__))
|
||||
|
||||
DEMO_ROUTE = "a2a0ccea32023010|2023-07-27--13-01-19"
|
||||
RELEASES_URL="https://github.com/commaai/PlotJuggler/releases/download/latest"
|
||||
INSTALL_DIR = os.path.join(juggle_dir, "bin")
|
||||
PLOTJUGGLER_BIN = os.path.join(juggle_dir, "bin/plotjuggler")
|
||||
MINIMUM_PLOTJUGGLER_VERSION = (3, 5, 2)
|
||||
MAX_STREAMING_BUFFER_SIZE = 1000
|
||||
|
||||
def install():
|
||||
m = f"{platform.system()}-{platform.machine()}"
|
||||
supported = ("Linux-x86_64", "Darwin-arm64", "Darwin-x86_64")
|
||||
if m not in supported:
|
||||
raise Exception(f"Unsupported platform: '{m}'. Supported platforms: {supported}")
|
||||
|
||||
if os.path.exists(INSTALL_DIR):
|
||||
shutil.rmtree(INSTALL_DIR)
|
||||
os.mkdir(INSTALL_DIR)
|
||||
|
||||
url = os.path.join(RELEASES_URL, m + ".tar.gz")
|
||||
with requests.get(url, stream=True, timeout=10) as r, tempfile.NamedTemporaryFile() as tmp:
|
||||
r.raise_for_status()
|
||||
with open(tmp.name, 'wb') as tmpf:
|
||||
for chunk in r.iter_content(chunk_size=1024*1024):
|
||||
tmpf.write(chunk)
|
||||
|
||||
with tarfile.open(tmp.name) as tar:
|
||||
tar.extractall(path=INSTALL_DIR)
|
||||
|
||||
|
||||
def get_plotjuggler_version():
|
||||
out = subprocess.check_output([PLOTJUGGLER_BIN, "-v"], encoding="utf-8").strip()
|
||||
version = out.split(" ")[1]
|
||||
return tuple(map(int, version.split(".")))
|
||||
|
||||
|
||||
def load_segment(segment_name):
|
||||
if segment_name is None:
|
||||
return []
|
||||
|
||||
try:
|
||||
return list(LogReader(segment_name))
|
||||
except (AssertionError, ValueError) as e:
|
||||
print(f"Error parsing {segment_name}: {e}")
|
||||
return []
|
||||
|
||||
|
||||
def start_juggler(fn=None, dbc=None, layout=None, route_or_segment_name=None):
|
||||
env = os.environ.copy()
|
||||
env["BASEDIR"] = BASEDIR
|
||||
env["PATH"] = f"{INSTALL_DIR}:{os.getenv('PATH', '')}"
|
||||
if dbc:
|
||||
env["DBC_NAME"] = dbc
|
||||
|
||||
extra_args = ""
|
||||
if fn is not None:
|
||||
extra_args += f" -d {fn}"
|
||||
if layout is not None:
|
||||
extra_args += f" -l {layout}"
|
||||
if route_or_segment_name is not None:
|
||||
extra_args += f" --window_title \"{route_or_segment_name}\""
|
||||
|
||||
cmd = f'{PLOTJUGGLER_BIN} --buffer_size {MAX_STREAMING_BUFFER_SIZE} --plugin_folders {INSTALL_DIR}{extra_args}'
|
||||
subprocess.call(cmd, shell=True, env=env, cwd=juggle_dir)
|
||||
|
||||
|
||||
def juggle_route(route_or_segment_name, segment_count, qlog, can, layout, dbc=None, ci=False):
|
||||
segment_start = 0
|
||||
if 'cabana' in route_or_segment_name:
|
||||
query = parse_qs(urlparse(route_or_segment_name).query)
|
||||
route_or_segment_name = query["route"][0]
|
||||
|
||||
if route_or_segment_name.startswith(("http://", "https://", "cd:/")) or os.path.isfile(route_or_segment_name):
|
||||
logs = [route_or_segment_name]
|
||||
elif ci:
|
||||
route_or_segment_name = SegmentName(route_or_segment_name, allow_route_name=True)
|
||||
route = route_or_segment_name.route_name.canonical_name
|
||||
segment_start = max(route_or_segment_name.segment_num, 0)
|
||||
logs = [get_url(route, i) for i in range(100)] # Assume there not more than 100 segments
|
||||
else:
|
||||
route_or_segment_name = SegmentName(route_or_segment_name, allow_route_name=True)
|
||||
segment_start = max(route_or_segment_name.segment_num, 0)
|
||||
|
||||
if route_or_segment_name.segment_num != -1 and segment_count is None:
|
||||
segment_count = 1
|
||||
|
||||
r = Route(route_or_segment_name.route_name.canonical_name, route_or_segment_name.data_dir)
|
||||
logs = r.qlog_paths() if qlog else r.log_paths()
|
||||
|
||||
segment_end = segment_start + segment_count if segment_count else None
|
||||
logs = logs[segment_start:segment_end]
|
||||
|
||||
if None in logs:
|
||||
resp = input(f"{logs.count(None)}/{len(logs)} of the rlogs in this segment are missing, would you like to fall back to the qlogs? (y/n) ")
|
||||
if resp == 'y':
|
||||
logs = r.qlog_paths()[segment_start:segment_end]
|
||||
else:
|
||||
print("Please try a different route or segment")
|
||||
return
|
||||
|
||||
all_data = []
|
||||
with multiprocessing.Pool(24) as pool:
|
||||
for d in pool.map(load_segment, logs):
|
||||
all_data += d
|
||||
|
||||
if not can:
|
||||
all_data = [d for d in all_data if d.which() not in ['can', 'sendcan']]
|
||||
|
||||
# Infer DBC name from logs
|
||||
if dbc is None:
|
||||
for cp in [m for m in all_data if m.which() == 'carParams']:
|
||||
try:
|
||||
DBC = __import__(f"openpilot.selfdrive.car.{cp.carParams.carName}.values", fromlist=['DBC']).DBC
|
||||
dbc = DBC[cp.carParams.carFingerprint]['pt']
|
||||
except Exception:
|
||||
pass
|
||||
break
|
||||
|
||||
with tempfile.NamedTemporaryFile(suffix='.rlog', dir=juggle_dir) as tmp:
|
||||
save_log(tmp.name, all_data, compress=False)
|
||||
del all_data
|
||||
start_juggler(tmp.name, dbc, layout, route_or_segment_name)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description="A helper to run PlotJuggler on openpilot routes",
|
||||
formatter_class=argparse.ArgumentDefaultsHelpFormatter)
|
||||
|
||||
parser.add_argument("--demo", action="store_true", help="Use the demo route instead of providing one")
|
||||
parser.add_argument("--qlog", action="store_true", help="Use qlogs")
|
||||
parser.add_argument("--ci", action="store_true", help="Download data from openpilot CI bucket")
|
||||
parser.add_argument("--can", action="store_true", help="Parse CAN data")
|
||||
parser.add_argument("--stream", action="store_true", help="Start PlotJuggler in streaming mode")
|
||||
parser.add_argument("--layout", nargs='?', help="Run PlotJuggler with a pre-defined layout")
|
||||
parser.add_argument("--install", action="store_true", help="Install or update PlotJuggler + plugins")
|
||||
parser.add_argument("--dbc", help="Set the DBC name to load for parsing CAN data. If not set, the DBC will be automatically inferred from the logs.")
|
||||
parser.add_argument("route_or_segment_name", nargs='?', help="The route or segment name to plot (cabana share URL accepted)")
|
||||
parser.add_argument("segment_count", type=int, nargs='?', help="The number of segments to plot")
|
||||
|
||||
if len(sys.argv) == 1:
|
||||
parser.print_help()
|
||||
sys.exit()
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.install:
|
||||
install()
|
||||
sys.exit()
|
||||
|
||||
if not os.path.exists(PLOTJUGGLER_BIN):
|
||||
print("PlotJuggler is missing. Downloading...")
|
||||
install()
|
||||
|
||||
if get_plotjuggler_version() < MINIMUM_PLOTJUGGLER_VERSION:
|
||||
print("PlotJuggler is out of date. Installing update...")
|
||||
install()
|
||||
|
||||
if args.stream:
|
||||
start_juggler(layout=args.layout)
|
||||
else:
|
||||
route_or_segment_name = DEMO_ROUTE if args.demo else args.route_or_segment_name.strip()
|
||||
juggle_route(route_or_segment_name, args.segment_count, args.qlog, args.can, args.layout, args.dbc, args.ci)
|
||||
148
tools/plotjuggler/layouts/camera-timings.xml
Normal file
148
tools/plotjuggler/layouts/camera-timings.xml
Normal file
@@ -0,0 +1,148 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab tab_name="SOF / EOF (encodeIdx)" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.500885;0.499115" count="2">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#1f77b4" name="/driverEncodeIdx/timestampSof">
|
||||
<transform name="Derivative" alias="/driverEncodeIdx/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#d62728" name="/roadEncodeIdx/timestampSof">
|
||||
<transform name="Derivative" alias="/roadEncodeIdx/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#1ac938" name="/wideRoadEncodeIdx/timestampSof">
|
||||
<transform name="Derivative" alias="/wideRoadEncodeIdx/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#f14cc1" name="/driverEncodeIdx/timestampEof">
|
||||
<transform name="Derivative" alias="/driverEncodeIdx/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#9467bd" name="/roadEncodeIdx/timestampEof">
|
||||
<transform name="Derivative" alias="/roadEncodeIdx/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#17becf" name="/wideRoadEncodeIdx/timestampEof">
|
||||
<transform name="Derivative" alias="/wideRoadEncodeIdx/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<Tab tab_name="model timings" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.5;0.5" count="2">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="0.015143" left="0.000000" top="0.016865" right="630.006367"/>
|
||||
<limitY/>
|
||||
<curve color="#ff7f0e" name="/modelV2/modelExecutionTime"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="-0.100000" left="0.000000" top="0.100000" right="630.006367"/>
|
||||
<limitY/>
|
||||
<curve color="#f14cc1" name="/modelV2/frameDropPerc"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<Tab tab_name="sensor info" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="1" count="1">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="-0.100000" left="0.000000" top="0.100000" right="630.006367"/>
|
||||
<limitY/>
|
||||
<curve color="#bcbd22" name="/driverCameraState/sensor"/>
|
||||
<curve color="#1f77b4" name="/roadCameraState/sensor"/>
|
||||
<curve color="#d62728" name="/wideRoadCameraState/sensor"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<Tab tab_name="SOF / EOF (cameraState)" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.500885;0.499115" count="2">
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#1f77b4" name="/driverCameraState/timestampSof">
|
||||
<transform name="Derivative" alias="/driverCameraState/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#d62728" name="/roadCameraState/timestampSof">
|
||||
<transform name="Derivative" alias="/roadCameraState/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#1ac938" name="/wideRoadCameraState/timestampSof">
|
||||
<transform name="Derivative" alias="/wideRoadCameraState/timestampSof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_x="false" mode="TimeSeries" flip_y="false" style="Lines">
|
||||
<range bottom="35000000.000000" left="0.000000" top="65000000.000000" right="630.006367"/>
|
||||
<limitY max="6.5e+07" min="3.5e+07"/>
|
||||
<curve color="#ff7f0e" name="/driverCameraState/timestampEof">
|
||||
<transform name="Derivative" alias="/driverCameraState/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#f14cc1" name="/roadCameraState/timestampEof">
|
||||
<transform name="Derivative" alias="/roadCameraState/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
<curve color="#9467bd" name="/wideRoadCameraState/timestampEof">
|
||||
<transform name="Derivative" alias="/wideRoadCameraState/timestampEof[Derivative]">
|
||||
<options lineEdit="1.0" radioChecked="radioCustom"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<customMathEquations/>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
72
tools/plotjuggler/layouts/can-states.xml
Normal file
72
tools/plotjuggler/layouts/can-states.xml
Normal file
@@ -0,0 +1,72 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget parent="main_window" name="Main Window">
|
||||
<Tab containers="1" tab_name="tab1">
|
||||
<Container>
|
||||
<DockSplitter count="2" sizes="0.500381;0.499619" orientation="-">
|
||||
<DockSplitter count="2" sizes="0.5;0.5" orientation="|">
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" mode="TimeSeries" flip_y="false" flip_x="false">
|
||||
<range right="632.799721" bottom="-17755.925000" top="771630.925000" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#1f77b4" name="/pandaStates/0/canState0/totalRxCnt"/>
|
||||
<curve color="#d62728" name="/pandaStates/0/canState1/totalRxCnt"/>
|
||||
<curve color="#1ac938" name="/pandaStates/0/canState2/totalRxCnt"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" mode="TimeSeries" flip_y="false" flip_x="false">
|
||||
<range right="632.799721" bottom="-18545.500000" top="760365.500000" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#ff7f0e" name="/pandaStates/0/canState0/totalTxCnt"/>
|
||||
<curve color="#f14cc1" name="/pandaStates/0/canState1/totalTxCnt"/>
|
||||
<curve color="#9467bd" name="/pandaStates/0/canState2/totalTxCnt"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
<DockSplitter count="3" sizes="0.333333;0.333333;0.333333" orientation="|">
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" mode="TimeSeries" flip_y="false" flip_x="false">
|
||||
<range right="632.799721" bottom="-1.350000" top="55.350000" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#ff7f0e" name="/pandaStates/0/canState0/totalRxLostCnt"/>
|
||||
<curve color="#f14cc1" name="/pandaStates/0/canState1/totalRxLostCnt"/>
|
||||
<curve color="#9467bd" name="/pandaStates/0/canState2/totalRxLostCnt"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" mode="TimeSeries" flip_y="false" flip_x="false">
|
||||
<range right="632.799721" bottom="-0.050000" top="2.050000" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#17becf" name="/pandaStates/0/canState0/totalTxLostCnt"/>
|
||||
<curve color="#bcbd22" name="/pandaStates/0/canState1/totalTxLostCnt"/>
|
||||
<curve color="#1f77b4" name="/pandaStates/0/canState2/totalTxLostCnt"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" mode="TimeSeries" flip_y="false" flip_x="false">
|
||||
<range right="632.799721" bottom="-0.100000" top="0.100000" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#17becf" name="/pandaStates/0/canState0/busOffCnt"/>
|
||||
<curve color="#1ac938" name="/pandaStates/0/canState1/busOffCnt"/>
|
||||
<curve color="#bcbd22" name="/pandaStates/0/canState2/busOffCnt"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
60
tools/plotjuggler/layouts/controls_mismatch_debug.xml
Normal file
60
tools/plotjuggler/layouts/controls_mismatch_debug.xml
Normal file
@@ -0,0 +1,60 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget parent="main_window" name="Main Window">
|
||||
<Tab tab_name="tab1" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" count="5" sizes="0.2;0.2;0.2;0.2;0.2">
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" flip_x="false" mode="TimeSeries" flip_y="false">
|
||||
<range top="1.025000" bottom="-0.025000" left="0.018309" right="59.674401"/>
|
||||
<limitY/>
|
||||
<curve color="#1f77b4" name="/controlsState/enabled"/>
|
||||
<curve color="#d62728" name="/pandaStates/0/controlsAllowed"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" flip_x="false" mode="TimeSeries" flip_y="false">
|
||||
<range top="27.087398" bottom="-0.905168" left="0.018309" right="59.674401"/>
|
||||
<limitY/>
|
||||
<curve color="#9467bd" name="/controlsState/cumLagMs"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" flip_x="false" mode="TimeSeries" flip_y="false">
|
||||
<range top="1.025000" bottom="-0.025000" left="0.018309" right="59.674401"/>
|
||||
<limitY/>
|
||||
<curve color="#1f77b4" name="/pandaStates/0/safetyRxInvalid"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" flip_x="false" mode="TimeSeries" flip_y="false">
|
||||
<range top="158.850000" bottom="-2.850000" left="0.018309" right="59.674401"/>
|
||||
<limitY/>
|
||||
<curve color="#d62728" name="/pandaStates/0/safetyTxBlocked"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot style="Lines" flip_x="false" mode="TimeSeries" flip_y="false">
|
||||
<range top="1.025000" bottom="-0.025000" left="0.018309" right="59.674401"/>
|
||||
<limitY/>
|
||||
<curve color="#1ac938" name="/carState/gasPressed"/>
|
||||
<curve color="#ff7f0e" name="/carState/brakePressed"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<customMathEquations/>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
104
tools/plotjuggler/layouts/demo.xml
Normal file
104
tools/plotjuggler/layouts/demo.xml
Normal file
@@ -0,0 +1,104 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root version="2.3.8">
|
||||
<tabbed_widget parent="main_window" name="Main Window">
|
||||
<Tab tab_name="tab1" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="1" count="1">
|
||||
<DockSplitter orientation="|" sizes="0.5;0.5" count="2">
|
||||
<DockSplitter orientation="-" sizes="0.500497;0.499503" count="2">
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" style="Lines">
|
||||
<range top="2.762667" bottom="-3.239397" right="56.512723" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#1f77b4" name="/carState/aEgo"/>
|
||||
<curve color="#17becf" name="/carState/brake"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" style="Lines">
|
||||
<range top="5.191867" bottom="-5.724069" right="56.512723" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#1ac938" name="dv/dt"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
<DockSplitter orientation="-" sizes="0.500497;0.499503" count="2">
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" style="Lines">
|
||||
<range top="16.065524" bottom="-0.470076" right="56.512723" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#d62728" name="/carState/vEgo"/>
|
||||
<curve color="#bcbd22" name="/carState/gas"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" style="Lines">
|
||||
<range top="1.014703" bottom="-0.012971" right="56.512723" left="0.000000"/>
|
||||
<limitY/>
|
||||
<curve color="#ff7f0e" name="/model/meta/brakeDisengageProb"/>
|
||||
<curve color="#f14cc1" name="/model/meta/engagedProb"/>
|
||||
<curve color="#9467bd" name="/model/meta/steerOverrideProb"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</DockSplitter>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad CSV">
|
||||
<default time_axis=""/>
|
||||
</plugin>
|
||||
<plugin ID="DataLoad ROS bags">
|
||||
<use_header_stamp value="false"/>
|
||||
<use_renaming_rules value="true"/>
|
||||
<discard_large_arrays value="true"/>
|
||||
<max_array_size value="100"/>
|
||||
</plugin>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="DataLoad ULog"/>
|
||||
<plugin ID="LSL Subscriber"/>
|
||||
<plugin ID="MQTT Subscriber"/>
|
||||
<plugin ID="ROS Topic Subscriber">
|
||||
<use_header_stamp value="false"/>
|
||||
<use_renaming_rules value="true"/>
|
||||
<discard_large_arrays value="true"/>
|
||||
<max_array_size value="100"/>
|
||||
</plugin>
|
||||
<plugin ID="UDP Server"/>
|
||||
<plugin ID="WebSocket Server"/>
|
||||
<plugin ID="ZMQ Subscriber"/>
|
||||
<plugin status="idle" ID="ROS /rosout Visualization"/>
|
||||
<plugin status="idle" ID="ROS Topic Re-Publisher"/>
|
||||
</Plugins>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<customMathEquations>
|
||||
<snippet name="dv/dt">
|
||||
<global>prevX = 0
|
||||
prevY = 0
|
||||
is_first = true</global>
|
||||
<function>if (is_first) then
|
||||
is_first = false
|
||||
prevX = time
|
||||
prevY = value
|
||||
end
|
||||
|
||||
dx = time - prevX
|
||||
dy = value - prevY
|
||||
prevX = time
|
||||
prevY = value
|
||||
|
||||
return dy/dx</function>
|
||||
<linkedSource>/carState/vEgo</linkedSource>
|
||||
</snippet>
|
||||
</customMathEquations>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
50
tools/plotjuggler/layouts/longitudinal.xml
Normal file
50
tools/plotjuggler/layouts/longitudinal.xml
Normal file
@@ -0,0 +1,50 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab tab_name="tab1" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" sizes="0.333601;0.332799;0.333601" count="3">
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" flip_x="false" mode="TimeSeries" style="Lines">
|
||||
<range right="60.000002" top="2.131164" left="0.000000" bottom="-3.377712"/>
|
||||
<limitY/>
|
||||
<curve name="/carState/aEgo" color="#f14cc1"/>
|
||||
<curve name="/longitudinalPlan/accels/0" color="#9467bd"/>
|
||||
<curve name="/carControl/actuators/accel" color="#17becf"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" flip_x="false" mode="TimeSeries" style="Lines">
|
||||
<range right="60.000002" top="19.304051" left="0.000000" bottom="-0.538805"/>
|
||||
<limitY/>
|
||||
<curve name="/carState/vEgo" color="#1ac938"/>
|
||||
<curve name="/longitudinalPlan/speeds/0" color="#ff7f0e"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" flip_x="false" mode="TimeSeries" style="Lines">
|
||||
<range right="60.000002" top="1.025000" left="0.000000" bottom="-0.025000"/>
|
||||
<limitY/>
|
||||
<curve name="/carControl/longActive" color="#1f77b4"/>
|
||||
<curve name="/carState/gasPressed" color="#d62728"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<customMathEquations/>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
92
tools/plotjuggler/layouts/max-torque-debug.xml
Normal file
92
tools/plotjuggler/layouts/max-torque-debug.xml
Normal file
@@ -0,0 +1,92 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab containers="1" tab_name="tab1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" count="4" sizes="0.249724;0.250829;0.249724;0.249724">
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" flip_x="false" mode="TimeSeries">
|
||||
<range left="0.000450" top="6.050533" right="2483.624998" bottom="-7.599037"/>
|
||||
<limitY/>
|
||||
<curve color="#1ac938" name="Actual lateral accel (roll compensated)"/>
|
||||
<curve color="#ff7f0e" name="Desired lateral accel (roll compensated)"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" flip_x="false" mode="TimeSeries">
|
||||
<range left="0.000450" top="5.384416" right="2483.624998" bottom="-7.503945"/>
|
||||
<limitY/>
|
||||
<curve color="#1ac938" name="roll compensated lateral acceleration"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" flip_x="false" mode="TimeSeries">
|
||||
<range left="0.000450" top="1.050000" right="2483.624998" bottom="-1.050000"/>
|
||||
<limitY/>
|
||||
<curve color="#0097ff" name="/carState/steeringPressed"/>
|
||||
<curve color="#d62728" name="/carControl/actuatorsOutput/steer"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" flip_x="false" mode="TimeSeries">
|
||||
<range left="0.000450" top="80.762969" right="2483.624998" bottom="-2.181837"/>
|
||||
<limitY/>
|
||||
<curve color="#f14cc1" name="/carState/vEgo">
|
||||
<transform alias="/carState/vEgo[Scale/Offset]" name="Scale/Offset">
|
||||
<options value_offset="0" time_offset="0" value_scale="2.23694"/>
|
||||
</transform>
|
||||
</curve>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<customMathEquations>
|
||||
<snippet name="roll compensated lateral acceleration">
|
||||
<global></global>
|
||||
<function>if (v3 == 0 and v4 == 1) then
|
||||
return (value * v1 ^ 2) - (v2 * 9.81)
|
||||
end
|
||||
return 0</function>
|
||||
<linked_source>/controlsState/curvature</linked_source>
|
||||
<additional_sources>
|
||||
<v1>/carState/vEgo</v1>
|
||||
<v2>/liveParameters/roll</v2>
|
||||
<v3>/carState/steeringPressed</v3>
|
||||
<v4>/carControl/latActive</v4>
|
||||
</additional_sources>
|
||||
</snippet>
|
||||
<snippet name="Desired lateral accel (roll compensated)">
|
||||
<global></global>
|
||||
<function>return (value * v1 ^ 2) - (v2 * 9.81)</function>
|
||||
<linked_source>/controlsState/desiredCurvature</linked_source>
|
||||
<additional_sources>
|
||||
<v1>/carState/vEgo</v1>
|
||||
<v2>/liveParameters/roll</v2>
|
||||
</additional_sources>
|
||||
</snippet>
|
||||
<snippet name="Actual lateral accel (roll compensated)">
|
||||
<global></global>
|
||||
<function>return (value * v1 ^ 2) - (v2 * 9.81)</function>
|
||||
<linked_source>/controlsState/curvature</linked_source>
|
||||
<additional_sources>
|
||||
<v1>/carState/vEgo</v1>
|
||||
<v2>/liveParameters/roll</v2>
|
||||
</additional_sources>
|
||||
</snippet>
|
||||
</customMathEquations>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
68
tools/plotjuggler/layouts/system_lag_debug.xml
Normal file
68
tools/plotjuggler/layouts/system_lag_debug.xml
Normal file
@@ -0,0 +1,68 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab tab_name="tab1" containers="1">
|
||||
<Container>
|
||||
<DockSplitter orientation="-" count="4" sizes="0.249729;0.250814;0.249729;0.249729">
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" flip_x="false" flip_y="false" style="Lines">
|
||||
<range top="102.500000" right="59.992103" left="0.000000" bottom="-2.500000"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/cpuUsagePercent/0" color="#1f77b4"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/1" color="#d62728"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/2" color="#1ac938"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/3" color="#ff7f0e"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/4" color="#f14cc1"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/5" color="#9467bd"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/6" color="#17becf"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/7" color="#bcbd22"/>
|
||||
<curve name="/deviceState/gpuUsagePercent" color="#1f77b4"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" flip_x="false" flip_y="false" style="Lines">
|
||||
<range top="64.005001" right="59.992103" left="0.000000" bottom="51.195000"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/cpuTempC/0" color="#d62728"/>
|
||||
<curve name="/deviceState/cpuTempC/1" color="#1ac938"/>
|
||||
<curve name="/deviceState/cpuTempC/2" color="#ff7f0e"/>
|
||||
<curve name="/deviceState/cpuTempC/3" color="#f14cc1"/>
|
||||
<curve name="/deviceState/cpuTempC/4" color="#9467bd"/>
|
||||
<curve name="/deviceState/cpuTempC/5" color="#17becf"/>
|
||||
<curve name="/deviceState/cpuTempC/6" color="#bcbd22"/>
|
||||
<curve name="/deviceState/cpuTempC/7" color="#1f77b4"/>
|
||||
<curve name="/deviceState/gpuTempC/0" color="#d62728"/>
|
||||
<curve name="/deviceState/gpuTempC/1" color="#1ac938"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" flip_x="false" flip_y="false" style="Lines">
|
||||
<range top="37.371108" right="59.992103" left="0.000000" bottom="-0.911490"/>
|
||||
<limitY/>
|
||||
<curve name="/modelV2/frameDropPerc" color="#f14cc1"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot mode="TimeSeries" flip_x="false" flip_y="false" style="Lines">
|
||||
<range top="-3.593455" right="59.992103" left="0.000000" bottom="-12.190956"/>
|
||||
<limitY/>
|
||||
<curve name="/controlsState/cumLagMs" color="#9467bd"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
107
tools/plotjuggler/layouts/thermal_debug.xml
Normal file
107
tools/plotjuggler/layouts/thermal_debug.xml
Normal file
@@ -0,0 +1,107 @@
|
||||
<?xml version='1.0' encoding='UTF-8'?>
|
||||
<root>
|
||||
<tabbed_widget name="Main Window" parent="main_window">
|
||||
<Tab containers="1" tab_name="tab1">
|
||||
<Container>
|
||||
<DockSplitter count="6" orientation="-" sizes="0.166785;0.166785;0.166075;0.166785;0.166785;0.166785">
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="87.987497" bottom="75.912497" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/cpuTempC/0" color="#1f77b4"/>
|
||||
<curve name="/deviceState/cpuTempC/1" color="#d62728"/>
|
||||
<curve name="/deviceState/cpuTempC/2" color="#1ac938"/>
|
||||
<curve name="/deviceState/cpuTempC/3" color="#ff7f0e"/>
|
||||
<curve name="/deviceState/cpuTempC/4" color="#f14cc1"/>
|
||||
<curve name="/deviceState/cpuTempC/5" color="#9467bd"/>
|
||||
<curve name="/deviceState/cpuTempC/6" color="#17becf"/>
|
||||
<curve name="/deviceState/cpuTempC/7" color="#bcbd22"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="85.861052" bottom="66.496950" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/pmicTempC/0" color="#1f77b4"/>
|
||||
<curve name="/deviceState/gpuTempC/0" color="#d62728"/>
|
||||
<curve name="/deviceState/gpuTempC/1" color="#1ac938"/>
|
||||
<curve name="/deviceState/memoryTempC" color="#f14cc1"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="86.207876" bottom="70.665918" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/maxTempC" color="#1f77b4"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="1.025000" bottom="-0.025000" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/thermalStatus" color="#1f77b4"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockSplitter count="3" orientation="|" sizes="0.333124;0.333752;0.333124">
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="12.057358" bottom="4.843517" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/powerDrawW" color="#ff7f0e"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="100.000000" bottom="0.000000" right="301.842654"/>
|
||||
<limitY min="0" max="100"/>
|
||||
<curve name="/deviceState/fanSpeedPercentDesired" color="#9467bd"/>
|
||||
<curve name="/pandaStates/0/fanPower" color="#1f77b4"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="5018.400000" bottom="255.600000" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/peripheralState/fanSpeedRpm" color="#1f77b4"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
<DockSplitter count="2" orientation="|" sizes="0.502513;0.497487">
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="100.025000" bottom="14.975000" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/cpuUsagePercent/0" color="#1f77b4"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/1" color="#d62728"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/2" color="#1ac938"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/3" color="#ff7f0e"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
<DockArea name="...">
|
||||
<plot flip_y="false" style="Lines" mode="TimeSeries" flip_x="false">
|
||||
<range left="0.006955" top="102.500000" bottom="-2.500000" right="301.842654"/>
|
||||
<limitY/>
|
||||
<curve name="/deviceState/cpuUsagePercent/4" color="#f14cc1"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/5" color="#9467bd"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/6" color="#17becf"/>
|
||||
<curve name="/deviceState/cpuUsagePercent/7" color="#bcbd22"/>
|
||||
</plot>
|
||||
</DockArea>
|
||||
</DockSplitter>
|
||||
</DockSplitter>
|
||||
</Container>
|
||||
</Tab>
|
||||
<currentTabIndex index="0"/>
|
||||
</tabbed_widget>
|
||||
<use_relative_time_offset enabled="1"/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
<Plugins>
|
||||
<plugin ID="DataLoad Rlog"/>
|
||||
<plugin ID="Cereal Subscriber"/>
|
||||
</Plugins>
|
||||
<customMathEquations/>
|
||||
<snippets/>
|
||||
<!-- - - - - - - - - - - - - - - -->
|
||||
</root>
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user