I am looking for a good replacement for INI files for storing large / complex configuration. So far I have used JvApplicatoinIniStorage + a custom INI file.
I am looking for a good replacement for INI files for storing large / complex configuration. So far I have used JvApplicatoinIniStorage + a custom INI file.
INI files have the advantage of being simple to read and edit by humans. Even though TMemIniFile removes the standard comments starting with semicolon, it does not remove those starting with e.g. //. They are simple to parse because of the TMemIniFile class and the above mentioned JVCL components.
Unfortunately when the configuration becomes more complex they get much more difficult to read and edit:
[Cameras]
count=4
[Cameras\Item0]
Name=Front Camera
CaptureFile=d:\capture\front.avi
// and much more
[Cameras\Item1]
Name=Left Macro Camera
CaptureFile=d:\capture\MacroLeft.avi
// and much more
[Cameras\Item2]
Name=Middle Macro Camera
CaptureFile=d:\capture\MacroMiddle.avi
// and much more
[Cameras\Item3]
Name=Right Macro Camera
CaptureFile=d:\capture\MacroRight.avi
// and much more
For a developer it is easy to understand that Count=4 means that there are Items 0 to 3, but that's a hurdle for others. And that's by far not the only problem.
And sometimes there are even more levels which makes it difficult to read even for developers:
[bla\blub\oink]
Also, the order of entries varies which makes it difficult to compare two files (that's why I wrote my IniFileFormatter tool)
https://sourceforge.net/projects/dzinifileformat/
I don't want to use the Registry because it is not portable and also it is not possible to easily compare different configurations.
I have considered JSON, but Warren Postma makes a pretty good point against it in
Edit: Wrong link: It wasn't Warren Postma but Martin Tournoij on:
https://arp242.net/weblog/json_as_configuration_files-_please_dont
Even though I like the approach much more than INI files.
XML is possible but not really human readable, also it lacks easy to use comments.
A database table would be an option but not for the kind of complex configuration we use.
Am I missing something? How do others solve it? Roll your own format? But there must already be a standard for such a common problem, mustn't there?
INI files have the advantage of being simple to read and edit by humans. Even though TMemIniFile removes the standard comments starting with semicolon, it does not remove those starting with e.g. //. They are simple to parse because of the TMemIniFile class and the above mentioned JVCL components.
Unfortunately when the configuration becomes more complex they get much more difficult to read and edit:
[Cameras]
count=4
[Cameras\Item0]
Name=Front Camera
CaptureFile=d:\capture\front.avi
// and much more
[Cameras\Item1]
Name=Left Macro Camera
CaptureFile=d:\capture\MacroLeft.avi
// and much more
[Cameras\Item2]
Name=Middle Macro Camera
CaptureFile=d:\capture\MacroMiddle.avi
// and much more
[Cameras\Item3]
Name=Right Macro Camera
CaptureFile=d:\capture\MacroRight.avi
// and much more
For a developer it is easy to understand that Count=4 means that there are Items 0 to 3, but that's a hurdle for others. And that's by far not the only problem.
And sometimes there are even more levels which makes it difficult to read even for developers:
[bla\blub\oink]
Also, the order of entries varies which makes it difficult to compare two files (that's why I wrote my IniFileFormatter tool)
https://sourceforge.net/projects/dzinifileformat/
I don't want to use the Registry because it is not portable and also it is not possible to easily compare different configurations.
I have considered JSON, but Warren Postma makes a pretty good point against it in
Edit: Wrong link: It wasn't Warren Postma but Martin Tournoij on:
https://arp242.net/weblog/json_as_configuration_files-_please_dont
Even though I like the approach much more than INI files.
XML is possible but not really human readable, also it lacks easy to use comments.
A database table would be an option but not for the kind of complex configuration we use.
Am I missing something? How do others solve it? Roll your own format? But there must already be a standard for such a common problem, mustn't there?
Correct me if I'm wrong, but I don't see an argument against JSON in the linked page? I see arguments against registry and I wholeheartedly agree with them.
ReplyDeleteFor simple, local, configurations I would use INI files. More complex ones JSON and configuration files that never need to be edited by the user (e.g. update information or application level configurations determined by the vendor) I'd use XML or JSON.
Where does Warren make a point against JSON in his article? Am I missing something?
ReplyDeleteSorry, the link was wrong. It wasn't Warren at all. I must have pasted the URL from a different tab I had open.
ReplyDeleteThe correct link is:
arp242.net - JSON as configuration files: please don’t
use DFM :)
ReplyDeleteobject Configuration: TConfiguration
object Camera1: TCamera
Name = 'Camera1'
CaptureFile = 'd:\capture\front.avi'
end
end
you can put almost anything in a DFM, you can "compile" it with Delphi as a ressource, and the containt do not need to exists at all, I mean the TConfiguration or TCamera objects do not need to exists (until you want to read the ressource as an object stream)
it just misses the ability to put comments (with the standard TParser)...I wonder if the compiler uses TParser also, it could handle comments an conditionnal compilation {$IFDEF ANDROID...} :)
I've made french paper about that (look at the code sections)
lookinside.free.fr - Look Inside
With kbmMW we have given the option to use one way to access configuration data, while providing multiple ways for the actual storage (ini, registry, XML, JSON etc).
ReplyDeletehttps://components4developers.blog/2017/09/16/rest-easy-with-kbmmw-7-configuration/
I am using XML, and it is enough for my needs. It has comments: w3.org - Extensible Markup Language (XML) 1.0 (Fifth Edition) And you are not forced to use such esoteric things as document types, shemas, etc. if you are not going to give your files to third parties for machine processing.
ReplyDeleteBSON as a variant.
ReplyDeleteblog.grijjy.com - Efficient and easy-to-use JSON and BSON library
Easy to use, automatic memory management, multiplatform, open source, records and objects serialization, BSON and JSON formats.
T n T as I understand it, BSON is a binary format. This would not allow editing the configuration with a text editor or at least a simple tool.
ReplyDeleteThomas Mueller The author doesn't give any advice on any alternative to JSON, so the article is of limited use except as an opinion. JSON isn't perfect (what is?), but by the sound of it, using JSON would be a sensible improvement over INI.
ReplyDeleteYAML is pretty readable
ReplyDeleteI'm simply saving my object into a JSON string.
ReplyDeleteI used this code (mormot is needed) :
unit System.configuration;
{$IFDEF FPC}
{$Mode DELPHI}
{$ENDIF}
interface
uses classes;
type
TConfigurationSettings = class(TObject)
public
class procedure LoadFromFile(Settings: TObject; filePath: string = '');
class procedure SaveToFile(Settings: TObject; filePath: string = '');
end;
implementation
uses
SysUtils, {$IFNDEF FPC}mormot, syncommons, IOUtils{$ELSE}fpjsonrtti {$ENDIF};
{ TConfigurationSettings }
class procedure TConfigurationSettings.LoadFromFile(Settings: TObject;
filePath: string = '');
var
strings: TStrings;
{$IFDEF FPC}
DeStreamer: TJSONDeStreamer;
{$ELSE}
Json: RawUTF8;
JsonPtr: PUTF8Char;
IsValidJson: Boolean;
{$ENDIF}
begin
try
{$IFDEF FPC}
DeStreamer := TJSONDeStreamer.Create(nil);
{$ENDIF}
if filePath = '' then
filePath := ChangeFileExt(ExtractFileName(paramstr(0)), '.json');
if FileExists(filePath) then
begin
strings := TStringList.Create;
strings.LoadFromFile(filePath);
{$IFDEF FPC}
DeStreamer.JSONToObject(strings.Text, Settings);
{$ELSE}
Json := strings.Text;
JsonPtr := @Json[1];
Settings := JSONToNewObject(JsonPtr, IsValidJson, []);
{$ENDIF}
strings.free();
end;
finally
{$IFDEF FPC}
DeStreamer.free;
{$ENDIF}
end;
end;
class procedure TConfigurationSettings.SaveToFile(Settings: TObject;
filePath: string = '');
var
strings: TStrings;
{$IFDEF FPC}
Streamer: TJSONStreamer;
{$ENDIF}
begin
{$IFDEF FPC}
Streamer := TJSONStreamer.Create(nil);
try
Streamer.Options := Streamer.Options + [jsoDateTimeAsString];
//jsoUseFormatString];
//Streamer.DateTimeFormat := 'yyyy-mm-dd"T"hh:mm:ss.zz';
{$ENDIF}
if filePath = '' then
filePath := ChangeFileExt(ExtractFileName(paramstr(0)), '.json');
strings := TStringList.Create;
try
{$IFDEF FPC}
strings.add(Streamer.ObjectToJSONString(Settings));
{$ELSE}
strings.add(ObjectToJSON(Settings, [woStoreClassName]));
{$ENDIF}
strings.SaveToFile(filePath);
finally
strings.free;
end;
{$IFDEF FPC}
finally
Streamer.free;
end;
{$ENDIF}
end;
end.
Thomas Mueller That library has support for BSON and JSON - you choose.
ReplyDeleteThomas Mueller
ReplyDeleteyou can use JSON, creating a class that will represent your configuration, and saving in this format.
See: https://community.embarcadero.com/blogs/entry/how-to-convert-an-object-to-json-and-back-with-a-single-line-of-code-497
The author of that article suggests an alternative in his related YAML post: github.com - toml-lang/toml
ReplyDelete"XML is possible but not really human readable"
ReplyDeleteI disagree with that. I replaced my old INIs by XML files, I don't think I've lost readbility within the files, they're still well structured (just "polluted" by a closing tag)
Nowadays I would switch directly to JSON anyways.
YAML is very readable... but a nightmare to parse :) kbmMW contains a full YAML implementation, which is also supported for kbmMWs configuration features.
ReplyDeleteYAML. yaml.org - YAML Ain't Markup Language
ReplyDeleteI use libyaml to parse YAML
ReplyDeleteRef drawbacks of Json as a config format.
ReplyDeleteI store configurations in the Registry without comments, and in the Database as records in a table - without comments. Both these are intended to be used as storage only - and changes happens programmatically. Json fits that pattern quite well.
Change log - if I needed one - would be through a VCS.
Do you really need an change log for a config file?
If you do, you already need a change management system - such as a VCS.
If you actually need informative text to guide hand editing, you can add more properties at each value node (description, format, default, etc), but hand editing any kind of hierarchical format is a risky sport. Lose a comma, a bracket of some sort, a colon, a quote - and the file is borked, regardless of format.
I have been through these hoops before. Switching to the registry, changing to XML (JSON and YAML simply wasn't at that time) - whatever I tried or suggested, the customers demanded to stay with INI files. The argument was merely: "We want a text file that can simply be copied and we are quite familiar with INI files."
ReplyDeleteDunno why JSON should be avoided. Visual Code uses JSON for configuration and settings extensively. It's nice. I mean to say, check out how it's done in Visual Code, it's quite inspiring.
ReplyDeleteI'm using JSON, never looked back at INI.
ReplyDeletea programs registry settings can be exported to a *.reg file that can then be double clicked on another setup to load those settings into the registry into that setup
ReplyDeleteJSON all the way. So much better that old school INI files. Also XML is way to verbose and no data types (everything is a string).
ReplyDelete