mirror of
https://github.com/richonguzman/LoRa_APRS_iGate.git
synced 2026-10-31 02:51:58 +01:00
Packet corruption #38
Assignees
adam (Adam Wojcieszonek)
Clear assignees
adam (Adam Wojcieszonek)
No Assignees
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: adam/LoRa_APRS_iGate#38
Reference in new issue
Block a user
Hi again Ricardo
I am getting an ususually high amount of packet corruption on my igate GW4BVE-10. An extract from the syslog file is attached. Some of the corrupted position reports are actually making it through APRS-IS to aprs.fi.
I think this may be related to a move to new hardware using the SX1262 LoRa chip. It is currently running on a T-Beam board which uses that chip. I don't think it happened on the earlier hardware.
I am happy to provide any more diagnostic information or perform any testing.
Regards and 73
John GW4BVE
All_2024-3-17-13 45 39_extract.csv
Hello. Can you provide exact board that you are using? I'm asking because as I remember correctly T-Beam uses SX1278 or SX1276 depending on the frequency. Maybe you can simply check on other board to confirm this problem? 73! Damian SQ2CPA
Hi, was watching the .csv file and this is really weird.
did you upload the ENVIRONMENT for this specific board? as for the data provided cannot validate if this is a packet corruption on the Tracker Tx or on the iGate Rx .
G7OMN-2 with old firmware SX126x radio not sending CRC. Therefore frequently corrupted packet. After the update on 24 February it now includes CRC.
Firstly it is only the igate that is using the SX1262. This is the board:
https://www.aliexpress.com/item/4001287221970.html
I selected the SX1262 and the M8N options. This board will eventually be used as a tracker but is currently in use as my igate GW4BVE-10.
Note that the corruption is there on the packets from a number of trackers including GW4BVE, MW0GWY, G7OMN. I know that the GW4BVE and MW0GWY trackers are NOT using the SX1262.
I will load the latest firmware on to my tracker this evening and report back.
Thanks & 73
John GW4BVE
A further thought. If G7OMN-2 is not sending a CRC on the packet then why is the igate accepting it as a valid packet. Is the CRC not being checked in the igate?
73
John GW4BVE
can you confirm you have the latest igate FW?
Before and after the CRC update, I spent weeks watching for bad packets. You can do this yourself at the following link. I'm sure it has something to do with the CRC checksum because after the update the number of errors dropped significantly. Even now there are still old firmware trackers and gateways that generate them. The link (you can't filter by date, only the last period)
The tocall is a good search solution (APLRG, APLRT)
http://www.findu.com/cgi-bin/errors.cgi?last=168
Another possible reason is that the maximum output power of the SX1278 is 20dBm, but above 17dBm there is a significant degradation. JGromes draws attention to this. Although neither the tracker nor the gateway uses radiolib the fact remains.
Hi Ricardo
The firmware is 08-03-2024
73 John GW4BVE
Consider that sx126x is more sensitive to frequency offset than sx127x.
sx127x can tolerate +/-12kHz or more, with sx126x above +/-4kHz CRC raise extremely.
I didn't know that, but a corrupted packet, even if it due to frequency offset, that fails the CRC test should not be accepted by the igate. The CRC is not perfect but this is a high number of errors which to me suggests that there is a CRC checking problem in the igate, possibly only when using the SX1262.
Its a mystery atm!!!
73 John GW4BVE
Agree in principle.
One thing to consider: the iGate accepts in principle frames in explicit and implicit mode. Usualle the tracker sends in explicit mode, and here coding rate, payload length as well as usage of crc is sent in the header.... so in theory if in explicit mode crc is sent to be not used the iGate would accept it.
So it would be interesting if you can identify tracker who produce crc and tracker which do not produce crc.
Are these trackers who produce crc also produce crc on other iGate ?
Do you face issues with signals on your rx frequency ? If you cannot rx the full packet without interference a crc will the resulting.
Check frequency offset as well as payload lengh as well as snr.... snr lower -18dB tend to crc, offset bigger 4kHz as well as payload length bigger 90byte tend to crc.
Thank you for your satisfactory and accurate reply! We seem to agree.
I have been concerned about this issue since January. 😉
Thanks for that explaination. I can confirm that I had corruption problems with packets from my tracker GW4BVE-5 running Ricardo's firmware 2023-08-26 and my igate running Ricardo's firmware 2024-03-08. So perhaps Ricardo can confirm if the CRC is enabled in the 2023-08-26 version of the tracker firmware.
I live in a very low interference environment but isn't this a distraction as corrupted packets should not be let through the igate? Either the CRC checking is disabled by the format of the packet header in the tracker or it is not being checked by the igate. I still suspect this is something to do with the SX1262 in the igate.
My tracker is now running the latest firmware from Ricardo and its callsign is G4BVE-3. I will see if I get corrupted packets with that. If I do I will change the igate to the orginal (non SX1262) board and test that combination.
73 John GW4BVE
the chip should not be the problem, but lets see how the test go on ;)
Hi Ricardo.
More information. I reviewed all my syslog yesterday and found that all trackers received by my igate had experienced packet corruption at some point. These are:
GW4BVE-5 - CA2RXU 2023-08026
G6UYG-12 - OE5BPA
G3SMT-3 - CA2RXU 2024-01-09
G6GVI-11 - OE5BPA
MW0GWY-3 - 2023-09-5
G7OMN-2 - CA2RXU 2024-01-26
G7NSJ-9 CA2RXU unknown version
G4OYX-4 - CA2RXU 2024-01-9
There is a good selection there, so the problem is not limited to one version of the tracker. I will go out today with my tracker which is now loaded with your latest firmware, then the next stage is to revert back to the board with the SX1278 chip for the igate and see what happens.
More information soon. Your help is much appreciated.
73 John GW4BVE
May I ask, do you also find error code -7 in the logs?
Hi
I uploaded the syslog with my original report. There does not seem to be an error code field in the syslog This is a typical corrupted packet log in the syslog:
2024-03-15,14:30:58,Notice,192.168.1.145,local4,1,"- GW4BVE-10 CA2RXU_LoRa_iGate_1.2 - - - Rx / <ÿMW0GWY-4>azþéd ,TzwJ* $/3aK|Ô@¹8®ÚrQ / -127dBm / -23.75dB / 3851Hz"
Where can I find the error codes?
73
John GW4BVE
In the log should be something like "status code -7"
This also indicates a CRC error.
HA5SZI Istvan
should we add to the syslog "log" also the "errors" ??
Here is an example. 😉
It might provide some diagnostic info but in my view packets that fail the CRC check should immediately be deleted. Presumably these are packets where the sync sequence has been detected but payload has been corrupted and therefore the CRC check will fail.
73 John GW4BVE
Hi Ricardo
I made a journey this morning with my tracker loaded with your latest firmware. I intentionally went to an area where the signal into my igate is marginal and I still get the errors. So we can conclude that the problem is present with many different versions of firmware on the trackers including your latest. I am now going to revert to an SX1278 based igate loaded with your latest firmware. Watch this space !!!!!
73 John GW4BVE
Hi Ricardo
Now running the Lilygo Lora 32 board with the SX1278 as the igate. I did another trip into a weak signal area and there was no packet corruption in the syslog. I have another journey planned tomorrow morning and will put an attenuator on the tracker to make the signal even weaker. It is looking like the problem is with the CRC checking on the SX1262 board used as an igate.
73 John GW4BVE
great info,... lets wait for your test to start making conclusions ;)
Hi Ricardo
I think the tests today will be conclusive. I received a Heltec V3 board (SX1262) yesterday and I am running it as GW4BVE-14. GW4BVE-10 is running on the old Lilygo LoRa 32 board (SX1278). They are on separate antennas at opposite ends of the house. Both are running your latest firmware.
73 John GW4BVE
Hi Ricardo
syslog attached.
GW4BVE-10 running Lilygo LoRa 32 (SX1278) board - 0 corrupted packets
GW4BVE-14 running Heltec V3 (SX1262) board - 8 corrupted packets
Both igates were running simulaneously on their own antennas
The packet corruption is the same on a T-Beam (SX1262) board, so it appears to be SX1262 related rather than a specific board.
Do you have enough information now? I am happy to do more tests if required.
73 John GW4BVE
GW4BVE_20240320.csv
John
thanks for the info, I dont see any detail that could provide some knowledge about what could be happening, as it seems just bad data decoding and no other way just this happens.... sometimes full packet error and sometimes just a letter here and there.
one more question:
are you using 433 modules? 915 modules?
I did a test last night, while the log server was running.
Gates:
HA5SZI-10=lora32 with SX1278 radio
HA5SZI-15=Heltec WSL with SX1262 radio.
Trackers:
HA5SZI-6=Heltec wireless tracker with SX1262 radio
HA5SZI-14=T-beam with SX1278 radio.
The trackers use a radiolib for all radios (including SX1278). Beacons are sent every 2 minutes all night long. There was no error in the log.
If I did that test with stationary and strong signals from the trackers I would not get any errors. The errors occur when the signals from the trackers are weak.....see my syslog Corrupted packets will be received by both SX1278 and SX1262 igates and that is perfectly normal with weak signals and they will have CRC failures. The difference is that the corrupted packets with CRC failures are being correctly dropped when using the SX1278 igate but are being accepted by the SX1262 igate.
Can you repeat that test with the trackers moving at the edge of your igate coverage?
73 John GW4BVE
Hi Ricardo
They are all 433MHz modules.
73 John GW4BVE
Correct. This is exactly what I would expect when the signal is weak and varying. The igate should drop the packets that are corrupted/badly decoded and the SX1278 igate does that well. The SX1262 igate lets the bad decodes through.
Do you check the CRC in your code or is it in a library? Does the low level code give an indication of CRC failure? Is the interface to the low level code different for the SX1262?
73 John GW4BVE
Hi again Ricardo
See section 6.2.3.5 CRC in the SX1262 datasheet. It looks to me as if the SX1262 igate is running in CRC_OFF (No CRC) mode.
73 John GW4BVE
Since I have been interested in this for a long time, I will try it out with a mobile tracker in the next few days. I haven't logged it so far but I've been watching the bugs for months. The gate doesn't cover a large area so I could do good tests with lower RF power from a shorter distance.
The gates also log RSSI and SNR values in telemetry (regardless of which one makes it up to the aprs-is). This could also be a good reference point.
Excellent. You plan is a good one and I will be interested to see the results.
Most of the corrupted frames do not make it through aprs-is to aprs.fi but some do. Those are the ones with small corruptions to just one field, say just a corrupted callsign or a corrupted lat/long. I have seen both of those on aprs.fi in fact that is where I first realised bad packets were not being detected and dropped on the SX1262 igate. It is much easier to see in the syslogs.
73 John GW4BVE
Off topic:
Just an interesting fact.
How well you can see the absence of TCXO. The upper part of the frequency error is SX1278, which follows the temperature deviation.
The lower band near the 0Hz line is SX1262.
On topic.
Hi Ricardo
I have done yet another test run. This time I have compared my GW4BVE-10 igate running on a Lilygo Lora 32 board running your latest firmware with the Heltec V3 board running OE5BPA latest firmware. The syslogs are attached.
The key thing to note is that the OE5BPA firmware correctly handles the packets that fail the CRC test. The error is noted in the syslog as: readData failed, code -7 but NO CORRUPTED PACKETS ARE SENT TO APRS-IS.
From the Radiolib documentation: [RADIOLIB_ERR_CRC_MISMATCH] The calculated and expected CRCs of received packet do not match. This means that the packet was damaged during transmission and should be sent again.
Your firmware seems not to handle this radiolib status code correctly and allows the corrupt packet with the CRC mismatch through to APRS-IS.
The syslog for this test is attached.
As always I am happy to to do any further testing you require.
73 John GW4BVE
GW4BVE_20240321.csv
It is simply disastrous. :D
aprstest.log
A bit extreme that. You are only confirming my tests that packets that fail the CRC are not being deleted in Ricardo's firmware. Of course the firmware should not be used until he fixes it but I don't believe it is a massive problem.
I do Python/Micropython programming and it is many years since I played Arduino but that was only on very small projects I am tempted to have a look at the code!!!
I am sure Ricardo will fix it.
73 John GW4BVE
I wanted to replace the lora32 with a heltec wireless stick lite. That's why I made a modified FW for it. This is still to wait., 🤣
@jclifford73 @Szetya
could you both run the "branch" --> radiolib revision ??
this has a fix proposal for this corruption case, and it should be solved... let us know how it goes
@richonguzman , @jclifford73
Well. The next test I wanted to try yesterday.
Tracker (Heltec wireless tracker XS1262) serial output as logger. Although all parts of the code are heavily modified it also uses radiolib. The other tracker is a T-beam, I set the TX power low enough (5dBm) to not have to go far with it.
The test seems to have been successful and lora_utils.cpp works fine.
(which as mentioned above is not the original code)
The next test will be the radiolib revision branch. ;)
logWithTracker.txt
aprs_igate_short.log
Hi Ricardo
The radiolib revision code still produces bad packets.
73 John GW4BVE
how can I mail you or telegram you?
Hi Ricardo
Email: johnc@jclifford.uk
I use Whatsapp not Telegram. Email me and I will give you my mobile number for Whatsapp. I need to put your mobile number in my contacts list.
73 John GW4BVE
Hi. Can you check latest dev version on the repo: https://github.com/SQ2CPA/LoRa_APRS_iGate Release is available.
I moved SX1278 from LoRa.h to Radiolib and I fixed some problem with SX126X.
Hi. I have this loaded on my test igate GW4BVE-17 which is running on a Heltec V3 board. I will be going to the edge of my coverage area, which is where the CRC errors occur. I will report back on the results.
73 John GW4BVE
The code in this branch does NOT solve the failure to detect CRC errors/packet corruption. The code in the radiolib-revision branch DOES solve the problem.
73 John GW4BVE
Ok, thank you for the test
@jclifford73 Please try again. Code from "radiolib-revision" was implemented. 73! Damian SQ2CPA
@SQ2CPA
If the radiolib generates an error then the loraPacket string "" is returned.
radio.startReceive(); << this does not need to be started every time it runs. It is enough once as well as after TX.
from radiolib:
// if needed, 'listen' mode can be disabled by calling
// any of the following methods:
//
// radio.standby()
// radio.sleep()
// radio.transmit();
// radio.receive();
// radio.scanChannel();
code snippet:
String receivePacket() {
String loraPacket = "";
if (receivedFlag) {
receivedFlag = false;
int state = radio.readData(loraPacket);
if (state == RADIOLIB_ERR_NONE) {
rssi = radio.getRSSI();
snr = radio.getSNR();
freqError = radio.getFrequencyError();
Utils::println("<--- LoRa Packet Rx : " + loraPacket);
Utils::println("(RSSI:" + String(rssi) + " / SNR:" + String(snr) + " / FreqErr:" + String(freqError) + ")");
if (Config.syslog.active && WiFi.status() == WL_CONNECTED && loraPacket != "") {
SYSLOG_Utils::log("Rx", loraPacket, rssi, snr, freqError);
}
} else if (state == RADIOLIB_ERR_RX_TIMEOUT) {
// timeout occurred while waiting for a packet
if (Config.syslog.active && WiFi.status() == WL_CONNECTED) {
SYSLOG_Utils::log("Rx", "ERROR TIMEOUT /", 0, 0, 0);
}
loraPacket = "";
} else if (state == RADIOLIB_ERR_CRC_MISMATCH) {
rssi = radio.getRSSI();
snr = radio.getSNR();
freqError = radio.getFrequencyError();
Serial.print("RSSI: " + String(rssi));
Serial.print(" ,SNR: " + String(snr));
Serial.print(" ,freqErr: " + String(freqError));
Serial.println("");
Serial.println(F("CRC error!"));
if (Config.syslog.active && WiFi.status() == WL_CONNECTED) {
SYSLOG_Utils::log("Rx", "ERROR CRC / " + loraPacket, rssi, snr, freqError);
}
loraPacket = "";
} else {
Serial.print(F("failed, code "));
Serial.println(state);
if (Config.syslog.active && WiFi.status() == WL_CONNECTED) {
SYSLOG_Utils::log("Rx", "ERROR Code:" + String(state), 0, 0, 0);
}
loraPacket = "";
}
}
return loraPacket;
}
Corrupted packets still there in the syslog with your version of the code. The code in Ricardo's radiolib-revision branch does not have this problem.
73 John GW4BVE
Do you mean now there are corrupted packet info in syslog ? Or corrupted packets being processed and shown on syslog?
As before the were no warning about crc packet fail
Hi Ricardo. I think there may be a bit of confusion here. Your fix in your radiolib-revision branch is fine. CRC errored packeds are detected and shown as RADIOLIB_ERR_CRC_MISMATCH. Is this in your main code now? My last message was re the code in https://github.com/SQ2CPA/LoRa_APRS_iGate which does NOT seem to work correctly.
73 John GW4BVE
please, could we talk only about last MAIN update?
the CRC sends RADIOLIB_ERR_CRC_MISMATCH to syslog and not to aprs-is, right?
sorry I've been a little confused with comparing different versions
please download latest fix just uploaded to test and report.
Hi Ricardo. All is OK. Time to close this issue I think. Many thanks for all your work and for sorting this out. 73 John GW4BVE
great news!