CSV-file format

CSV-file format was created by Mats

Posted 5 years 2 days ago #191962
I'm a new user of the Pscan diagnostic tool. Living in Sweden and have an MGF VVC 1997.
Was trying to use the log feature, but came into problems with data format when importing to Excel.
Data delimiter is a comma-sign. In continental Europe we use decimal comma in stead of decimal point so data import gets crapped.
There is a workaround by changing PC settings to english, but would be much better if the data delimiter was changed to some other character (eg. semi-colon).
Thanks.
by Mats

Please Log in or Create an account to join the conversation.

Replied by talkingcars on topic CSV-file format

Posted 5 years 1 day ago #191963
Hi and welcome to the forum

You may be able to find the comma as a special character, have a look and let us know.

James


Home to black Alfa Romeo 159 3.2 V6 Q4 ,green MGF VVC and red MG Maestro T16.

MG - the friendly marque.

Please Log in or Create an account to join the conversation.

Replied by pscan.uk on topic CSV-file format

Posted 5 years 1 day ago #191965
Instead of double clicking on the csv file do this:-

Start Excel (or Libreoffice).
Start a new blank spreadsheet.
Look for import csv (in current Excel it's Data -> From text/csv, in Libre Office file -> open).

When you do it this way Excel / Libreoffice allows you to select the delimiter. Select comma and desect all other characters.

Then it will work.

I have gone to a lot of effort to make sure that commas do not appear in any text output in live data. If you find one please report it. I really don't want to change to another delimiter as this means re-visiting thousands of lines of code for dozens of ECUs.

It just isn't the most effective use of my time when there is a working way of importing the data in Excel and Libreoffice.

Please Log in or Create an account to join the conversation.

Replied by Mats on topic CSV-file format

Posted 5 years 1 day ago #191974
I do import the csv-file exactly as you advice.
My point is that in all continental Europe (including Germany, France and Spain) we (and all Windows applications) use comma as the decimal delimiter and Pscan seem to be using standard Windows settings for this.
Example below is live data for Injector Firing Interval,Injector pulsewidth,Injector Deadtime,Fueling feedback correction bank 1,
csv-file from english log: ,720.0 °,2.1 ms,0.8 ms,100.0 %,
csv-file from non-english log: ,720,0 °,2,1 ms,0,8 ms,100,0 %,

The workaround is ok for me but I have to change Windows settings each time I use Pscan and reset after.
I presume your customers from continental Europe haven't used the logging feature yet, but could be good to inform them about the workaround.

Apart from this I'm very pleased with the Pscan tool. It helps a lot to understand engine behavior.
Thanks
Mats
by Mats
The following user(s) said Thank You: gerry

Please Log in or Create an account to join the conversation.

Replied by pscan.uk on topic CSV-file format

Posted 5 years 1 day ago #191975
I will think about the best solution. Temporarily you can open the csv file in notepad and do search/replace and change all the commas to semicolons (or whatever you prefer) , save and then open in excel. This might be easier than reconfiguring windows.

Please Log in or Create an account to join the conversation.

Replied by pscan.uk on topic CSV-file format

Posted 5 years 1 day ago #191976
It's strange that RFC4180 says nothing about this.
According to this article https://en.wikipedia.org/wiki/Delimiter I should consider using non printable ascii character 30 (Hex 1E) as the delimiter.

Please could you take an output file from pscan, open it with a Hex editor (I use HxD), change all the commas (Hex 2C) to Hex 1E, save and load into Excel please.

and then tell me if that works

Please Log in or Create an account to join the conversation.

Replied by pscan.uk on topic CSV-file format

Posted 5 years 1 day ago #191977
Also I have a German engineer next to me at the moment who says that the delimiter should be a tab character (ascii code 9) which strictly speaking means that the file becomes a TSV (tab separated value) rather than CSV.

So that's something else to try.

It will make it harder to "preprocess" files in notepad before importing into Excel though.

Update: tab seems to not be a problem for notepad, and tab separate text is one of the normal options for excel when "export" data is used so I am leaning towards using tab.
Last Edit:5 years 1 day ago by pscan.uk
Last edit: 5 years 1 day ago by pscan.uk.

Please Log in or Create an account to join the conversation.

Replied by Mats on topic CSV-file format

Posted 5 years 1 day ago #191978
I have looked into the csv but a replace would also replace the decimal commas, so not a working solution.
For the time being I will use my workaround.
by Mats

Please Log in or Create an account to join the conversation.

Replied by pscan.uk on topic CSV-file format

Posted 5 years 1 day ago #191979
I don't understand.

Are you saying that the original csv file produced by pscan produces decimal numbers with a comma instead of dot?

I don't believe that it does.

You should be able to take a csv file produced by pscan and replace the commas with tabs without this destroying the decimal places in the data.

Please Log in or Create an account to join the conversation.

Replied by Mats on topic CSV-file format

Posted 5 years 1 day ago #191980
That is what I understand. The tool seem to use windows settings for how various things are displayed.
I presume you have declared what character set you are using. I get some other strange character also.
Attached my first log file (renamed txt as csv was not allowed.
by Mats

Please Log in or Create an account to join the conversation.

Replied by pscan.uk on topic CSV-file format

Posted 5 years 1 day ago #191981
Which ECU were you connected to when you created this log file please?

Please Log in or Create an account to join the conversation.

Replied by Mats on topic CSV-file format

Posted 5 years 1 day ago #191984
Having a VVC I connected to MEMS2J
by Mats

Please Log in or Create an account to join the conversation.

Moderators: pscan.uk
Time to create page: 0.531 seconds