ApplicationProperties and multi processes

General JUCE discussion
1

Could you add something like a file modification detector for the PropertiesFile to reload the file automatically when it changed, or before it will be written.
This would help to remove inconsistency in data, when an application with multiple processes sharing the same property file (My app uses processes for instances for high availability/stability)

2

That’s something that I’ve thought about that over the years, but it’s a trickier problem than it sounds - to really avoid inconsistency you’d end up pretty much writing an object-database!

3

i now what you mean.
You could create a “lock”-File (like in the same path as the properties file with a “.lock” postfix) where you need to get write-access, even if you only read the file, so you can simulate a atomic access.
And if the modification-date changed since the last access, reload it, before any write or read access. That should fit for 99.999% of all possibilities, and improves the current situation.

ps:
If you don’t get “write access” to the lock file after a second, you could ignore it (and jassert - as a fallback, only to prevent a freeze while waiting for the lock)

4

Did you notice that PropertiesFile already has support for interprocess locking? It won’t check for other processes making changes, but you can at least avoid the file getting corrupted when being written.

5

cool,

i added a second virtual function to ProperySet: virtual void checkForModifications()
And call that before every read & and write.

In PropertyFile i overwrite checkForModifications to check for the the last modification (and may reload) and added a line in propertyChanged() to save the last modification date.
Very easy, seems to work (and it would even better if the modification-check also uses the lock seamless with the write)

[attachment=0]checkForModificationPatch.zip[/attachment]

[code]Index: modules/juce_core/containers/juce_PropertySet.cpp

— modules/juce_core/containers/juce_PropertySet.cpp (revision 2941)
+++ modules/juce_core/containers/juce_PropertySet.cpp (working copy)
@@ -54,8 +54,9 @@
void PropertySet::clear()
{
const ScopedLock sl (lock);

  • if (properties.size() > 0)
  • checkForModifications();
  • if (properties.size() > 0)
    {
    properties.clear();
    propertyChanged();
    @@ -63,9 +64,10 @@
    }

String PropertySet::getValue (const String& keyName,

  •                          const String& defaultValue) const noexcept

  •                          const String& defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();

    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

@@ -77,9 +79,10 @@
}

int PropertySet::getIntValue (const String& keyName,

  •                          const int defaultValue) const noexcept

  •                          const int defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();
    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

    if (index >= 0)
    @@ -90,9 +93,10 @@
    }

double PropertySet::getDoubleValue (const String& keyName,

  •                                const double defaultValue) const noexcept

  •                                const double defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();
    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

    if (index >= 0)
    @@ -103,9 +107,10 @@
    }

bool PropertySet::getBoolValue (const String& keyName,

  •                            const bool defaultValue) const noexcept

  •                            const bool defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();
    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

    if (index >= 0)
    @@ -115,14 +120,16 @@
    : defaultValue;
    }

-XmlElement* PropertySet::getXmlValue (const String& keyName) const
+XmlElement* PropertySet::getXmlValue (const String& keyName)
{
return XmlDocument::parse (getValue (keyName));
}

void PropertySet::setValue (const String& keyName, const var& v)
{

  • jassert (keyName.isNotEmpty()); // shouldn’t use an empty key name!

  • checkForModifications();

  • jassert (keyName.isNotEmpty()); // shouldn’t use an empty key name!

    if (keyName.isNotEmpty())
    {
    @@ -141,6 +148,8 @@

void PropertySet::removeValue (const String& keyName)
{

  • checkForModifications();
  • if (keyName.isNotEmpty())
    {
    const ScopedLock sl (lock);
    @@ -156,6 +165,7 @@

void PropertySet::setValue (const String& keyName, const XmlElement* const xml)
{
+
setValue (keyName, xml == nullptr ? var::null
: var (xml->createDocument (String::empty, true)));
}
@@ -218,3 +228,8 @@
void PropertySet::propertyChanged()
{
}
+
+void PropertySet::checkForModifications()
+{
+
+}
Index: modules/juce_core/containers/juce_PropertySet.h

— modules/juce_core/containers/juce_PropertySet.h (revision 2941)
+++ modules/juce_core/containers/juce_PropertySet.h (working copy)
@@ -71,7 +71,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
String getValue (const String& keyName,

  •                 const String& defaultReturnValue = String::empty) const noexcept;

  •                 const String& defaultReturnValue = String::empty)  noexcept;

    /** Returns one of the properties as an integer.

@@ -83,7 +83,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
int getIntValue (const String& keyName,

  •                 const int defaultReturnValue = 0) const noexcept;

  •                 const int defaultReturnValue = 0)  noexcept;

    /** Returns one of the properties as an double.

@@ -95,7 +95,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
double getDoubleValue (const String& keyName,

  •                       const double defaultReturnValue = 0.0) const noexcept;

  •                       const double defaultReturnValue = 0.0)  noexcept;

    /** Returns one of the properties as an boolean.

@@ -110,7 +110,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
bool getBoolValue (const String& keyName,

  •                   const bool defaultReturnValue = false) const noexcept;

  •                   const bool defaultReturnValue = false)  noexcept;

    /** Returns one of the properties as an XML element.

@@ -123,7 +123,7 @@

     @param keyName              the name of the property to retrieve
 */
  • XmlElement* getXmlValue (const String& keyName) const;

  • XmlElement* getXmlValue (const String& keyName) ;

    //==============================================================================
    /** Sets a named property.
    @@ -199,10 +199,18 @@
    /
    PropertySet     getFallbackPropertySet() const noexcept { return fallbackProperties; }

protected:
/** Subclasses can override this to be told when one of the properies has been changed. */
virtual void propertyChanged();

  • /** Subclasses can override for tasks before reading or writing values */
  • virtual void checkForModifications() ;

private:
StringPairArray properties;
PropertySet* fallbackProperties;
Index: modules/juce_data_structures/app_properties/juce_PropertiesFile.cpp

— modules/juce_data_structures/app_properties/juce_PropertiesFile.cpp (revision 2941)
+++ modules/juce_data_structures/app_properties/juce_PropertiesFile.cpp (working copy)
@@ -350,4 +350,16 @@
startTimer (options.millisecondsBeforeSaving);
else if (options.millisecondsBeforeSaving == 0)
saveIfNeeded();
+

  • lastModificationTime=file.getLastModificationTime();
    }

+void PropertiesFile::checkForModifications()
+{

  • if (lastModificationTime!=file.getLastModificationTime())

  • {

  •   reload();

  •   lastModificationTime=file.getLastModificationTime();

  • }
    +}
    Index: modules/juce_data_structures/app_properties/juce_PropertiesFile.h
    ===================================================================
    — modules/juce_data_structures/app_properties/juce_PropertiesFile.h (revision 2941)
    +++ modules/juce_data_structures/app_properties/juce_PropertiesFile.h (working copy)
    @@ -222,9 +222,14 @@
    /** @internal */
    virtual void propertyChanged();

  • /** @internal */

  • virtual void checkForModifications();

private:
//==============================================================================
File file;

  • Time lastModificationTime;
  • Options options;
    bool loadedOk, needsWriting;[/code]
6

Sorry, can immediately see lots of things I don’t like about those changes, and don’t have time to spend on it right now, will have to return to it some other time!

7

yes i now, i assume you don’t like the remove of the constness of the get methods…
…the modificationDate update must also be updated directly in the save functions…done
and yes, the file access (for mod date) with every read you may think that’s not smart, but the only way to have integrity…
and yes, the lock has to be used seamlessly for checking the modification date and for the read/write after, but than we have to change the architecture of propertyset/file…
Anyway for those who need a quick solution, which is working better than now it terms of integrity, here is an updated patch

[code]Index: modules/juce_core/containers/juce_PropertySet.cpp

— modules/juce_core/containers/juce_PropertySet.cpp (revision 2941)
+++ modules/juce_core/containers/juce_PropertySet.cpp (working copy)
@@ -54,8 +54,8 @@
void PropertySet::clear()
{
const ScopedLock sl (lock);

  • if (properties.size() > 0)
  • if (properties.size() > 0)
    {
    properties.clear();
    propertyChanged();
    @@ -63,9 +63,10 @@
    }

String PropertySet::getValue (const String& keyName,

  •                          const String& defaultValue) const noexcept

  •                          const String& defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();

    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

@@ -77,9 +78,10 @@
}

int PropertySet::getIntValue (const String& keyName,

  •                          const int defaultValue) const noexcept

  •                          const int defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();
    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

    if (index >= 0)
    @@ -90,9 +92,10 @@
    }

double PropertySet::getDoubleValue (const String& keyName,

  •                                const double defaultValue) const noexcept

  •                                const double defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();
    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

    if (index >= 0)
    @@ -103,9 +106,10 @@
    }

bool PropertySet::getBoolValue (const String& keyName,

  •                            const bool defaultValue) const noexcept

  •                            const bool defaultValue)  noexcept

{
const ScopedLock sl (lock);

  • checkForModifications();
    const int index = properties.getAllKeys().indexOf (keyName, ignoreCaseOfKeys);

    if (index >= 0)
    @@ -115,14 +119,16 @@
    : defaultValue;
    }

-XmlElement* PropertySet::getXmlValue (const String& keyName) const
+XmlElement* PropertySet::getXmlValue (const String& keyName)
{
return XmlDocument::parse (getValue (keyName));
}

void PropertySet::setValue (const String& keyName, const var& v)
{

  • jassert (keyName.isNotEmpty()); // shouldn’t use an empty key name!

  • checkForModifications();

  • jassert (keyName.isNotEmpty()); // shouldn’t use an empty key name!

    if (keyName.isNotEmpty())
    {
    @@ -141,6 +147,8 @@

void PropertySet::removeValue (const String& keyName)
{

  • checkForModifications();
  • if (keyName.isNotEmpty())
    {
    const ScopedLock sl (lock);
    @@ -156,6 +164,7 @@

void PropertySet::setValue (const String& keyName, const XmlElement* const xml)
{
+
setValue (keyName, xml == nullptr ? var::null
: var (xml->createDocument (String::empty, true)));
}
@@ -218,3 +227,8 @@
void PropertySet::propertyChanged()
{
}
+
+void PropertySet::checkForModifications()
+{
+
+}
Index: modules/juce_core/containers/juce_PropertySet.h

— modules/juce_core/containers/juce_PropertySet.h (revision 2941)
+++ modules/juce_core/containers/juce_PropertySet.h (working copy)
@@ -71,7 +71,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
String getValue (const String& keyName,

  •                 const String& defaultReturnValue = String::empty) const noexcept;

  •                 const String& defaultReturnValue = String::empty)  noexcept;

    /** Returns one of the properties as an integer.

@@ -83,7 +83,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
int getIntValue (const String& keyName,

  •                 const int defaultReturnValue = 0) const noexcept;

  •                 const int defaultReturnValue = 0)  noexcept;

    /** Returns one of the properties as an double.

@@ -95,7 +95,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
double getDoubleValue (const String& keyName,

  •                       const double defaultReturnValue = 0.0) const noexcept;

  •                       const double defaultReturnValue = 0.0)  noexcept;

    /** Returns one of the properties as an boolean.

@@ -110,7 +110,7 @@
@param defaultReturnValue a value to return if the named property doesn’t actually exist
*/
bool getBoolValue (const String& keyName,

  •                   const bool defaultReturnValue = false) const noexcept;

  •                   const bool defaultReturnValue = false)  noexcept;

    /** Returns one of the properties as an XML element.

@@ -123,7 +123,7 @@

     @param keyName              the name of the property to retrieve
 */
  • XmlElement* getXmlValue (const String& keyName) const;

  • XmlElement* getXmlValue (const String& keyName) ;

    //==============================================================================
    /** Sets a named property.
    @@ -199,10 +199,18 @@
    /
    PropertySet     getFallbackPropertySet() const noexcept { return fallbackProperties; }

protected:
/** Subclasses can override this to be told when one of the properies has been changed. */
virtual void propertyChanged();

  • /** Subclasses can override for tasks before reading or writing values */
  • virtual void checkForModifications() ;

private:
StringPairArray properties;
PropertySet* fallbackProperties;
Index: modules/juce_data_structures/app_properties/juce_PropertiesFile.cpp

— modules/juce_data_structures/app_properties/juce_PropertiesFile.cpp (revision 2941)
+++ modules/juce_data_structures/app_properties/juce_PropertiesFile.cpp (working copy)
@@ -241,6 +241,9 @@
return true;
}

  • lastModificationTime=file.getLastModificationTime();
  • return false;
    }

@@ -264,6 +267,7 @@
}
}

  • return false;
    }

@@ -332,6 +336,8 @@
}
}

  • lastModificationTime=file.getLastModificationTime();
  • return false;
    }

@@ -350,4 +356,16 @@
startTimer (options.millisecondsBeforeSaving);
else if (options.millisecondsBeforeSaving == 0)
saveIfNeeded();
+
+
}
+
+void PropertiesFile::checkForModifications()
+{

  • if (lastModificationTime!=file.getLastModificationTime())

  • {

  •   reload();

  •   lastModificationTime=file.getLastModificationTime();

  • }
    +}
    Index: modules/juce_data_structures/app_properties/juce_PropertiesFile.h
    ===================================================================
    — modules/juce_data_structures/app_properties/juce_PropertiesFile.h (revision 2941)
    +++ modules/juce_data_structures/app_properties/juce_PropertiesFile.h (working copy)
    @@ -222,9 +222,14 @@
    /** @internal */
    virtual void propertyChanged();

  • /** @internal */

  • virtual void checkForModifications();

private:
//==============================================================================
File file;

  • Time lastModificationTime;
  • Options options;
    bool loadedOk, needsWriting;

[/code]

8

what about saving every parameter in its own file? (as third mode, beside xml/binary) (using the Identifier as part of the filename).

MyApp_colour.value
MyApp_defaultPath.value
MyApp_thingsToRember.value

That would protect parameters, if you overwrite other parameters.

9

That’s great if you only have 10 values, but that could get messy!

TBH what I’d really like to do with the whole class is to scrap it and come up with a better solution based on ValueTrees. Would be really cool to have a ValueTree that’s shared between processes and automatically stored in a file. It’d involve some tricky synchronisation work, but would be really neat.

10

Yeah like a database. All nice.( Does the Undo-Manager already save "delta-information?!)
But a little way over-the-top for my use-case i need now (just in the moment)

BTW i use ValueTrees all the way for network communication (Thats why i ask you to extend the var type to store MemoryBlocks, cause i don’t want another Layer of abstraction, to store ANY information (yes inclusive sample data!)) in ValueTrees!
Because there VT are stream-able in to the memory, its a perfect tool to get over the “process-memory barrier” (a lack of c++, because it has no reflection)